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Overview 


In 2007, New York City launched the first test of a conditional cash transfer program in the United 
States. Called Family Rewards, the program sought to break the intergenerational cycle of poverty 
by offering cash assistance to poor fa mi lies to reduce immediate hardship, but conditioned this 
assistance on fa mi lies’ efforts to improve their health, further their children’s education, and increase 
parents’ work and earnings, in the hope of reducing poverty over the long term. The program had 
positive effects on some outcomes, but left others unchanged. Building on the lessons learned from 
that evaluation led to the next iteration and test of the model — called Family Rewards 2.0, the 
subject of this report. 

Family Rewards 2.0 was launched in July 2011 in the Bronx, New York and Memphis, Tennessee. 
While still offering rewards in the areas of children’s education, family health, and parents’ work. 
Family Rewards 2.0 has fewer rewards in each domain, offers the education rewards only to high 
school students, makes the rewards more timely by paying them each month, and includes family 
guidance. The addition of guidance, or having staff members actively help fa mi lies develop strate- 
gies to earn rewards, represents the biggest change to the original model. 

MDRC is evaluating the program through a randomized controlled trial involving approximately 
1,200 fa mi lies in each city, half of whom can receive the cash rewards if they meet the required 
conditions, and half of whom have been assigned to a control group that cannot receive the rewards. 
This report presents early findings on the program’s implementation and fa mi lies’ receipt of rewards 
during the first two years. 


Key Findings 

After some recmitment and start-up challenges in Year 1, the program was operating generally as 
envisioned in both cities by Year 2. The findings indicate that: 

• Nearly all fa mi lies earned at least some rewards, and the average family who earned rewards 
earned $2,160 during Year 2. 

• The family guidance component evolved substantially over time, moving from less intensive 
interactions focused on paperwork in Year 1 to more intensive interactions designed to help 
families take steps to earn rewards in Year 2. 

• Fa mi lies in Family Rewards 2.0 seem to have understood the rewards more completely than 
families did in 1 .0, and they were more likely to earn the rewards that were offered. 

• The revised model, and most probably the more intensive family guidance component, succeed- 
ed in engaging the types of fa mi lies who were less engaged in Family Rewards 1.0. 

• However, offering fewer rewards meant that the total amount of cash transferred to fa mi lies was 
less in Family Rewards 2.0 than in Family Rewards 1 .0. 

The final assessment of whether the new program is more effective than the original model will come 
in a later report, with the impact analysis. But the findings to date on the first two years of implementa- 
tion are encouraging and suggest that Family Rewards 2.0 was a step in the right direction in the effort 
to assess whether the conditional cash transfer model can work in the United States. 
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Preface 


Conditional cash transfer programs offer cash assistance to poor families, but condition this 
assistance on families’ efforts to improve their health and further their children’s education. 
They are fairly common in low- and middle-income countries, but until seven years ago they 
had never been tested in the United States. That is when MDRC began to test Opportunity 
NYC: Family Rewards (which this report refers to as Family Rewards 1.0), a three-year 
demonstration sponsored by the New York City Center for Economic Opportunity. MDRC 
helped design the initiative based on several existing international programs, notably Mexico’s 
Oportunidades, and conducted the impact and implementation evaluations. Low-income 
families were offered cash incentives (“rewards”) for completing activities related to children’s 
education, family preventive health care, and adult work or training. The program had moderate 
effects, but did reduce hardship, increase access to dental care, and improve the educational 
outcomes of more academically prepared high school students. 

The findings from that first demonstration were encouraging enough to convince sever- 
al stakeholders that it should be revised and tried again, this time with the help of the Social 
Innovation Fund. This is an example of the kind of work MDRC is committed to: developing 
programs of research related to particular social problems or intervention ideas that build 
progressively on existing evidence. 

This new version of Family Rewards reduces the number of rewards and eliminates ed- 
ucation rewards for elementary and middle school students. It extends the program to Memphis, 
providing an opportunity to test the concept in a local context very different from New York, 
restricts eligibility to families receiving food stamps or cash assistance, and adds a family 
guidance component. Family Rewards 2.0 thus shifted from a test of a large set of incentives 
with little support to one of a smaller set of incentives combined with active family guidance. 
Testing this version of the program solely with families receiving either cash or food assistance 
also starts to simulate what a conditional cash transfer program could look like if it were 
embedded in the social services system of the United States. 

This report shows that the program was implemented well and participation rates were 
high. At the same time, it is clear that the program underwent numerous changes, most notably 
in the area of family guidance, where motivational interviewing was incorporated in the second 
year. This raises the question of whether the attempted improvements to the program occurred 
early enough to have a positive effect. The next report will assess the program’s effects, 
describing Family Rewards 2.0’s impacts on education, health, poverty, and employment. 

Gordon L. Berlin 
President 
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Executive Summary 


In 2007, New York City launched the first test of a conditional cash transfer program in the 
United States. Called Opportunity NYC-Family Rewards, or Family Rewards for short, the 
program sought to break the intergenerational cycle of poverty by offering cash assistance to 
poor families to reduce immediate hardship, but conditioned this assistance on parents’ efforts 
to complete activities related to their children’s education, their family’s health care, and their 
work, in the hope of reducing poverty over the longer term. Conditional cash transfer programs 
have become widely used in low- and medium-income countries and have generally been 
successful at reducing poverty and increasing schooling and health care use. ' 

The evaluation of Family Rewards showed that the program reduced poverty in the 
short term and led to some improvements in children’s education, health care use, and parents’ 
work. While the program’s effects in each domain were either limited to subgroups or modest in 
size. Family Rewards led to enough positive changes to suggest that it could be made more 
effective if modified somewhat or aimed at certain types of families.^ The lessons learned from 
that evaluation led to the next iteration and test of the model — called Family Rewards 2.0. 

Family Rewards 2.0, the subject of this report, was launched in July 201 1 in the Bronx, 
New York and Memphis, Tennessee. While still offering rewards in the areas of children’s 
education, family health, and parents’ work. Family Rewards 2.0 is a refinement of the original 
model in several ways, offering fewer rewards in each domain, restricting the education rewards 
to high school students, and offering guidance to help families earn rewards. The addition of 
guidance, or having staff members actively help families develop strategies to earn rewards, 
represents the biggest change to the original model. Funded by the Social Innovation Fund of 
the Corporation for National and Community Service and private organizations, the demonstra- 
tion is managed by The Mayor’s Fund to Advance New York City, New York City’s Center for 
Economic Opportunity, and MDRC. The Children’s Aid Society (CAS) is operating the 
program in partnership with two community-based organizations in each city. MDRC is 
conducting the evaluation. 


'Fiszbein, Ariel, and Norbert Schady, Conditional Cash Transfers: Reducing Present and Future Poverty 
(Washington, DC: The World Bank, 2009). 

^Riccio, James, Nadine Dechausay, David Greenberg, Cynthia Miller, Zawadi Rucks, and Nandita Verma, 
Toward Reduced Poverty Across Generations: Early Findings from New York City’s Conditional Cash 
Transfer Program (New York: MDRC, 2010); Riccio, James, Nadine Dechausay, Cynthia Miller, Stephen 
Nunez, Nandita Verma, and Edith Yang, Conditional Cash Transfers in New York City: The Continuing Story 
of the Opportunity NYC-Family Rewards Demonstration (New York: MDRC, 2013). 
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This report presents early findings on the implementation of the new design of Family 
Rewards and on families’ receipt of rewards in the second year. It begins to address the question 
of whether the refined and simplified model was an improvement over the original program by 
assessing how it operated, how families understood and engaged with it, and how family 
guidance can fit within a traditional conditional cash transfer model. A later report, examining 
program impacts, will assess whether the new model ultimately led to bigger effects on educa- 
tion, health, and work. Finally, the report also compares the implementation of the program in 
the two cities. The Bronx and Memphis represent two very different contexts; it is important to 
assess whether a program of this type can be put in place under different circumstances, and if 
so how. 


The findings indicate that, after some recmitment and start-up challenges in Year 1, the 
program was operating generally as envisioned in both cities by the middle of Year 2. The 
family guidance component evolved considerably over time, starting out as fairly modest and 
transactional during Year 1 and becoming more intensive during Year 2. Families earned on 
average more than $2,000 in rewards in Year 2. There were no big differences in implementa- 
tion between the two cities, although differences in local context led to some modifications in 
how the program was delivered. Finally, the changes to the Family Rewards 1.0 model seem to 
have led to increased understanding and earning among families. A later report will assess 
whether they ultimately led to larger impacts. 


The Program Model 

Like its predecessor (and other conditional cash transfer programs). Family Rewards 2.0 is 
based on the assumption that for a variety of reasons families may underinvest in their own 
development even though such investments can have long-term benefits. Financial incentives 
can help change their calculations, encouraging them to make extra investments of time and 
energy in certain educational, health care, and work-related efforts. Furthermore, the extra 
resources can help make it more feasible for low-income people to undertake such efforts in the 
short term, by helping them pay for educational materials or tutoring for children, for example, 
or transportation to a free dental clinic, or clothes for a job interview. In this sense, the condi- 
tional rewards may function not only as incentives to action but also as enabling resources. 

The original Family Rewards model offered families 22 cash rewards, covering activ- 
ities and outcomes in three domains. In the education domain, rewards were offered for 
parents’ attendance at parent-teacher conferences and for children’s attendance in school, 
performance on standardized tests, completion of adequate credits per year, passing of exams 
required to graduate, and graduation. In the health domain, rewards were offered for main- 
taining health coverage and for preventive medical and dental checkups for each family 
member. In the work domain, parents were offered rewards for sustaining full-time work and 
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for pursuing education or training while working. Families received payments for rewards 
earned every two months. Families were not offered services or counseling, since the goal 
was to test a pure incentives model. 

During the three years the program operated, the average participating family earned 
nearly $9,000 in rewards, or roughly $3,000 in each year, leading to large reductions in poverty. 
The program did not affect school outcomes for elementary or middle school students, but did 
improve outcomes for the ninth-graders in the study who were performing at a proficient level 
or better academically when they entered, with sizable effects on grade promotion and on 
graduation. In the health area, early, positive effects on visits to the doctor and health status 
faded, although there were continued impacts on health coverage and, especially, dental visits. 
Finally, the program led to modest increases in employment throughout the follow-up period. 

Family Rewards 2.0 includes rewards for the following milestones (see Table ES.l): 

• Education: Students are rewarded for high attendance, good grades, perfor- 
mance on state core exams, and taking college entrance exams. 

• Health: Families receive payments for obtaining medical and dental check- 
ups for each family member. 

• Work and training: Parents receive payments for full-time work and for 
earning General Educational Development (GED) certificates. 

The program makes several important modifications to the original Family Rewards 
model. In an effort to make the program easier to understand and focus families’ attention on a 
limited number of outcomes, the program offers 8 rewards across the three domains, instead of 
22 rewards. While all children in a family are eligible to earn health rewards, the education 
rewards are only offered to high school students, given that the original program had no effects 
for younger students. The new program attempts to make the rewards more timely, and thus 
more salient to families, in two ways: first, by paying families monthly for rewards earned, 
rather than every two months; and second, by rewarding students for grades earned. The 
rewards for grades have also been stmctured in an attempt to engage less academically profi- 
cient students, with rewards offered for A, B, and C grades. 

Most notably, however, the new model offers guidance to help families earn rewards. 
The addition of this component is the biggest innovation to the Family Rewards 2.0 model. The 
original model asked staff members to provide advice and referrals to participants when asked 
for assistance with services, but did not require them to reach out actively to families to engage 
them in conversations about strategies to earn rewards. Findings from the first evaluation 
suggested that many families needed more help to reach the relevant milestones. Under the new 
model, staff members at the community-based partner organizations develop a Family Earning 
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SIF Family Rewards 
Table ES.l 


Family Rewards 2,0: Schedule of Rewards 


Domain 

Bronx 

Memphis 

Education incentives 

High school students only 



Attends 95% of scheduled 
school days 

$40 per month 

$40 per month 

Takes an SAT or ACT exam 
(once during program) 

$50 

$50; must score 19 or more 
on the ACT if administered 
by Memphis City Schools'* 

Receives grades on an official 
report card*” 

$30 per A (90-100); $20 per 
B (80-89); $10 per C (75-79) 

$30 per A (93-100); $20 per 
B (85-92); $10 per C (75-84) 

Passes up to 5 Regents exams'^ 
or 7 End of Course exams'^ 

$500 per Regents exam for a 
score of 75 or above; $400 
per exam for a score of 65-74 

$200 per End of Course exam 
for a score of proficient or 
advanced (increased to $300 
in Year 2) 

Health incentives 

Parents and children 19 and younge 

r 


Annual medical checkup 

$100 per family member 

$100 per family member 

Preventive dental care every 
6 months (once per year for 
children 1-5 years old) 

$100 per family member per 
visit 

$100 per family member per 
visit 

Workforce incentives 

Parents only 



Sustains full-time employment 

$150 per month 

$150 per month 

Earns a GED certificate 

$400 

$400 


(continued) 
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Table ES.l (continued) 


SOURCE: CAS's Family Rewards program materials. 

NOTES: GED = General Educational Development. 

“The ACT is scored out of 36. Memphis City Schools officials requested that the minimum score for 
the reward be set at 19 for students taking the test for free in class because this is considered an indication 
that students are ready for college-level work. 

'’In Year 1, the amount of each reward for grades was prorated based on the number of official report 
cards issued by a student's school. To simplify verification in Year 2, students were paid the listed 
amounts for their grades regardless of the number of official report cards they received, up to a maximum 
of $600 per program year. 

“High school students (grades 9-12) in the Bronx were eligible to earn rewards for the following Regents 
exams: English, one of any math exams (including Math A, Math B, Integrated Algebra, Geometry, and 
Algebra 2/Trigonometry), U.S. History and Government, Global History and Geography, and one of any 
science exams (including Living Environment, Chemistry, Physics, and Earth Science). 

“^High school students (grades 9-12) in Memphis were eligible to earn rewards for the following End of 
Course exams: Algebra 1, Algebra 2, Biology, English 1, English 2, English 3, and U.S. History. 


Plan with every family and meet with them twice per year to discuss their progress toward 
earning the rewards. Staff members are also directed to conduct more aggressive outreach to 
families who are not earning rewards. 

CAS is managing the operations of Family Rewards 2.0, providing technical assistance 
and oversight to four neighborhood partner organizations (NPOs), two in each city, selected to 
implement the program. The NPOs are charged with implementing core components of the 
program. They recmited and enrolled families into the research sample, oriented families to the 
program, and provide continuing guidance to help families earn rewards. These groups serve as 
the face of the program in the communities served. Families receive payments for meeting 
reward milestones, in most cases by submitting “coupons” to CAS to verify that the milestones 
were met, in some cases with supporting documents. Two of the rewards, for attendance and for 
passing state core exams, are automatically verified using school records, requiring no action on 
the part of the family to earn the payment. 


The Study Sample and the Recruitment Process 

Family Rewards 2.0 is being evaluated using a randomized controlled trial. In each city, about 
1,200 families were recmited for the study. Half were randomly assigned to a program group, 
eligible for Family Rewards, and half were assigned to a control group, not eligible for the 
program. Rewards are offered for three years, and the evaluation will track families for several 
years to determine the effects of the program on poverty, children’s education, family health, 
and parents’ work. 
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The program targeted families with at least one child entering ninth grade or tenth grade. 
Once enrolled in Family Rewards 2.0, however, all of the family’s children under age 20 became 
eligible for the health-related rewards. The program also targeted recipients of benefits from 
Temporary Assistance for Needy Families (TANF) and the Supplemental Nutrition Assistance 
Program (food stamps), in order to get resources to the neediest families and to explore how 
conditional cash transfer concepts might be integrated into these safety-net programs. 

Recmitment began in August 2011 in New York and September 2011 in Memphis, 
when the NPOs received lists of potentially eligible participants provided by the human services 
agencies and departments of education in each city. The pace of recmitment varied between the 
cities due to delays in receiving some lists and difficulties in reaching some potential partici- 
pants. Enrollment was originally expected to conclude by October 2011, but in practice the 
majority of families in the study enrolled after that point, and many enrolled as late as January 
or Febmary 2012. For this reason, this report focuses on reward receipt during the second 
program year, when all families were eligible to earn a full year of rewards. 

The majority of families who enrolled in the study in both cities were single-parent 
families, with a higher percentage in Memphis (90 percent) than in New York (77 percent). A 
large fraction of the sample in New York is of Hispanic origin (74 percent), while nearly all 
participating families in Memphis are African American (98 percent). Adults in New York had 
somewhat lower education levels when they enrolled than adults in Memphis. For example, 
nearly half of the New York adults did not have either a GED certificate or high school diploma 
when they enrolled in the study, compared with only 31 percent of adults in Memphis. In 
contrast, adults in the New York sample were more likely to be working when they entered the 
study (57 percent) than their counterparts in Memphis (44 percent). 


Implementation and Reward Receipt 

While this version of a conditional cash transfer model built on the previous version, it was still 
challenging to implement, given its varied components and the fact that it was being put in 
place in a new city. The report documents that after some challenges at the start, many of which 
are typical of new programs, by the middle of Year 2 the program was fully functioning as 
planned. Recmitment delays led to some challenges in Year 1, and the family guidance compo- 
nent required additional monitoring and training to become more intensive by Year 2. 

• Parents and teenagers had a good understanding of the rewards and the 
verification procedures, although they did experience a fair number of 
coupon rejections during the first two years. 
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Because the program is intended to shape everyday decisions that participants make in 
relation to education, health, and work, it is critical that they remember and understand all of the 
rewards offered, how to earn them, and how to claim them. Families learned about the rewards 
in the initial program orientation sessions, at which they received coupon books for earning 
rewards, and were reminded about them through mailings and through Earnings Statements that 
detailed rewards earned for the month. While data from the 24-month survey will provide a 
more complete picture of how families understood the program, focus groups with a subset of 
families in the study suggest that they understood the program well and could identify the 
available rewards and the steps needed to claim them. 

Despite a fairly high level of program knowledge, many families experienced at least 
one rejection of a submitted coupon. About 70 percent of families in New York experienced at 
least one rejection, as did 92 percent of families in Memphis. The most common rejections were 
of coupons claiming rewards for full-time work and for report card grades, as students attempt- 
ed to claim rewards for classes or grades that were not eligible. Although the criterion for 
earning the work reward was straightforward — working 120 hours per month — participants 
often had difficulty with the verification procedures. The most common reasons for rejections of 
this coupon were: too few hours, missing pay stubs, or pay stubs that did not align with the 
program schedule. Nonetheless, the rejections do not appear to have affected families’ level of 
engagement with the program, and participants do not seem to have become discouraged or 
frustrated after learning that a coupon had been rejected. 

• Nearly all families earned at least some rewards, and the average family 
who earned rewards earned $2,160 during Year 2. 

As mentioned above, this report focuses on rewards earned in Year 2, since that is the 
first full year in which all families were enrolled and were eligible to earn rewards. As shown in 
Table ES.2, most families (96 percent) earned at least some rewards in Year 2, and those who 
earned at least some rewards earned on average $2,160 over the year ($2,281 in New York and 
$2,043 in Memphis). Overall reward earnings came largely from the education and health areas: 
93 percent of families earned at least one education reward, 79 percent earned at least one health 
reward, and 44 percent earned at least one work reward. 

There was some variation among families in the amount received, and families who 
earned more rewards differed in several ways from families who earned less. Families receiving 
the most money included parents who, at the time they entered the study, were more educated, 
more likely to be working full time, and more likely to be married than parents in other families. 

• The family guidance component evolved substantially over time, moving 
from less intensive interactions focused on paperwork in Year 1 to more 
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SIF Family Rewards 
Table ES.2 

Summary of Rewards Earned by Families in Year 2 


Outcome 

Bronx 

Memphis 

Total 

Family earned at least 1 reward (%) 

94.8 

97.9 

96.3 

Education reward 

89.8 

96.4 

93.1 

Attendance 

82.7 

94.6 

88.6 

State core exam 

68.9 

56.6 

62.8 

SAT/ACT 

9.1 

3.3 

6.2 

Report card 

78.0 

82.4 

80.2 

Health reward 

79.1 

78.5 

78.8 

Annual physical 

72.3 

68.5 

70.4 

Biannual dental 

70.0 

71.5 

70.7 

Workforce reward 

45.5 

41.9 

43.7 

Full-time employment 

45.4 

41.3 

43.3 

GED certificate 

0.5 

1.1 

0.8 

Among families who earned at least 1 reward 

Average reward amount earned'^ ($) 

2,281 

2,043 

2,160 

Percentage of earnings from student rewards ’’ 

58.1 

56.1 

57.1 

Average reward amount earned, by domain ($) 

Education 

1,287 

1,014 

1,146 

Health 

617 

706 

661 

Worhforce 

1,140 

1,117 

1,129 

Family picked up coupon book (%) 

95.5 

94.5 

95.0 

Sample size 

617 

613 

1,230 


SOURCE: MDRC calculations using CAS's Family Rewards program data. 

NOTES: Sample sizes may vary because of missing values. 

Year 2 covers reward activities that occurred between September 2012 and August 2013. 
Italics indicate outcomes calculated for a subset of the full sample. 

“The lowest and highest amounts earned in Year 2 were $38 and $7,250 for the Bronx 
sample, $40 and $6,680 for the Memphis sample, and $38 and $7,250 for the full sample. 

'’Percentage of earnings from student rewards is calculated using education rewards earned 
by high school students among families who earned rewards and had students who earned 
education rewards in Year 2. 
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intensive interactions designed to help families take steps to earn re- 
wards in Year 2. 


The biggest innovation in the design of Family Rewards 2.0 was the addition of the 
family guidance component. The program designers believed that families would benefit from a 
regular source of support within the program, but aimed to stmcture that support in a way that 
fit within a conditional cash transfer model. The designers created a “triage” model of guidance 
in which all participants receive some guidance, but those who are having the most trouble 
earning rewards receive targeted, more intensive support. The model was also designed so that 
it could be implemented by a team of paraprofessionals. 

In Year 1, the guidance interaction was focused on building relationships and providing 
customer service. The staff tried to build tmsting, familiar relationships with parents and 
students, and to ensure that participants understood the incentives and how to claim them. These 
activities fell short of the model design, since the Advisors were not targeting those participants 
who earned fewer rewards, nor were they engaged in deep conversations about barriers to 
earning rewards. 

Recognizing this, a form of counseling called motivational interviewing was introduced 
into family guidance at the start of Year 2. Motivational interviewing is a widely used, directive 
approach to counseling in which a counselor uses a set of techniques to explore an individual’s 
ambivalence about change in an effort to get him or her to express a commitment to change. It 
is a client-driven approach in which the individual is viewed as the expert in changing his or her 
life. It took months of training for the staff to achieve a minimum level of competence in 
motivational interviewing, yet observations reveal that the nature and content of family guid- 
ance sessions did change significantly from Year 1 to the end of Year 2. In addition, the 
Advisors began targeting the participants earning the fewest rewards in Year 2, a group that 
proved to be challenging to serve. 


Implementation Across Cities 

• The program was implemented well in both cities, although differences 
in local context required adaptations by NPOs to stay connected with 
participants. 

In general, the program was implemented similarly across both cities and across the 
four neighborhood partner organizations. In some cases, however, local circumstances affected 
the implementation of certain components. In particular, in Memphis participants had chal- 
lenges related to transportation. This affected NPOs’ ability to recmit families and the quality 
of their interaction with families once enrolled. When families did not show up for guidance 
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meetings, for example, the staff adapted by conducting guidance over the phone or making 
home visits. Although this was a reasonable response, the unintended result was that Memphis 
staff members spent less time having intensive one-on-one meetings with participants and 
seemed to have less close relationships with them. Similarly, many students could not get to an 
NPO on their own, so parents tended to drop off or bring in report cards for them, which meant 
the Advisors did not develop personal relationships with many of the students. This was 
reflected in participants’ views: in both locations they reported that they valued their relation- 
ships with Advisors highly, but this sentiment was expressed more often in the Bronx than in 
Memphis. It is not clear if this difference between cities will lead to different outcomes for 
participants, but it is an important part of the context that affected program delivery. 

• Families in both cities were fairly involved in earning rewards, although 
families in Memphis earned somewhat less than families in the Bronx. 

Implementing the Family Rewards model in both New York and Memphis provides an 
important test of whether the model can operate with different populations and in different local 
contexts, and if so how. Focus groups indicated that families in both cities understood the 
program well. Overall, families who earned rewards earned similar amounts in the two cities, 
although families in Memphis earned slightly less on average in Year 2 ($2,043) than families 
in the Bronx ($2,281). Reward receipt rates did differ, however, for specific rewards. For 
example, students in the Bronx were less likely to earn rewards for attendance because in New 
York tardiness is counted toward absences. In contrast, more students in New York earned the 
high-value reward for the state core exams because they were well informed about the exams 
and took practice tests in class as preparation. In Memphis, these exams were relatively new. 

Finally, fewer adults claimed the reward for dental checkups in Memphis than in New 
York, largely because the public health insurance program in Tennessee does not cover dental 
cleanings, as it does in New York. Many adults in Memphis cited the out-of-pocket cost of 
dental care as the reason they did not earn that reward. 


Family Rewards 2.0 Compared with 1.0 

• Fam ili es in Family Rewards 2.0 seem to have understood the rewards 
more completely than families in 1.0, and they were more likely to earn 
rewards. 

Parents and students generally had no trouble remembering the incentives when asked 
to list them several months after orientation. In most cases, the biggest challenge was remem- 
bering the specific dollar value associated with the reward, particularly for rewards for which 
the amounts changed between Year 1 and Year 2 (this happened for grades, for example) or for 
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rewards that were paid out infrequently (for example, those for taking college prep tests). 
Parents also understood the work rewards fairly well, knowing the hour minimum and the 
documents that needed to be submitted in order to earn them. Although it is difficult to attribute 
parents’ improved understanding to a particular program feature, it seems as though reducing 
the number of rewards had a positive effect on families’ understanding of the program, as 
intended. 

Another indication of better understanding is the higher rates of reward receipt. When 
comparing receipt of individual rewards that were offered in both programs, families in Family 
Rewards 2.0 were more likely to earn almost every reward. Students had higher receipt rates for 
attendance rewards, for example, and passing state core exams. The relatively high use of these 
rewards was probably due to the program’s greater emphasis of them, given the shorter list of 
activities, and the Advisors’ marketing of them and support for earning them. 

• However, offering fewer rewards meant that the total amount of cash 
transferred to families was less in Family Rewards 2.0 than in 1.0. 

Families in the original Family Rewards program who earned rewards earned on aver- 
age over $3,000 per year during the first two years of the program. Families in that study who 
were similar to families in the new program (for example, those with high school students and 
those receiving TANF or food stamps) had higher average earnings among those who earned 
rewards in Year 2, at about $3,500. In contrast, families in Family Rewards 2.0 who earned 
rewards earned on average about $2,200 in Year 2. 

The elimination of the reward for health insurance was a big factor contributing to the 
difference. In the original program, families could earn up to $100 every month for maintaining 
health insurance for their children and themselves, and receipt rates were fairly high for this 
reward given the high rates of coverage. Another reason for the difference is the elimination of 
rewards for elementary and middle school students: younger siblings in the new program did 
not contribute to family earnings via education. Finally, the education rewards offered to high 
school students were somewhat more generous in Family Rewards 1.0 than they are in 2.0, 
which also contributed to higher earnings for the original program. 

Another result of the changed reward stmcture is that the earnings from students now 
make up a much higher fraction of total family earnings. Student earnings made up 57 percent 
of total family earnings in Family Rewards 2.0, compared with 33 percent in Family Rewards 
1.0. This difference in the source of family earnings may change how the program affects 
family dynamics and family material well-being, particularly if students’ earnings are viewed as 
separate from the family budget. The lower overall level of rewards earned also suggests that 
effects on poverty and material hardship, to be examined in a later report, may be more modest. 
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• When fully in place in Year 2, the revised model, and most probably the 
more intensive family guidance component, succeeded in increasing re- 
ward rates among the types of families who earned the least in Family 
Rewards 1.0. 

In Family Rewards 1.0, the families who earned the most from the program were on 
average less disadvantaged than other families. The higher-earning families had adults with 
higher levels of education, employment, earnings from employment, marriage, and self-reported 
mental and physical health than did the lower-earning families. This pattern was also evident for 
program impacts, with larger effects on school outcomes for more proficient students and more 
positive effects on employment for adults with higher education levels. 

The designers of the new program were particularly interested in improving participation 
and program impacts for two subgroups at risk of earning few rewards: adults who did not have a 
high school diploma or GED certificate at the start of the study and students who were not 
academically proficient at the start of the study. ^ Data through Year 2 indicate that the program 
has succeeded in increasing the rate at which these groups earn rewards. For example, among 
students who did not score at the proficient level on their English tests before they entered the 
study, 87 percent earned at least one attendance reward and 54 percent earned at least one reward 
for passing a state core exam in Family Rewards 2.0. The corresponding rates in Family Rewards 
1.0 were 55 percent and 39 percent (for high school students in families receiving TANF or food 
stamps). As another example, among parents with lower education levels, 38 percent earned at 
least one reward for full-time work in Family Rewards 2.0 compared with 26 percent in Family 
Rewards 1 .0. This higher level of reward receipt for more disadvantaged families suggests that 
the new program is an improvement over the original model. 

Of course, the final assessment of whether the new program is more effective than the 
original model will come in a later report with the impact analysis. But the findings to date on 
the first two years of implementation are encouraging, and suggest that Family Rewards 2.0 was 
a step in the right direction in the effort to assess whether the conditional cash transfer model 
can work in the United States. The research will follow the program and control groups for the 
next several years to assess the program’s impacts on income, well-being, education, health, and 
work. An impact report covering the first three years will be completed in 2015. 


^In 2010, the New York State Department of Education raised the scores necessary to be deemed “profi- 
cient.” As a result, proficiency rates will be much lower for the Family Rewards 2.0 Bronx sample than they 
were in the Family Rewards 1.0 sample. The findings reported here were similar when proficiency was 
determined for the Family Rewards 1.0 sample using the new, higher score requirements. 
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Chapter 1 

Introduction 


Conditional cash transfer programs are fairly common in lower- and middle-income countries. 
Designed to break the intergenerational cycle of poverty, these programs offer cash assistance to 
poor families, but typically condition this assistance on families’ efforts to improve their health 
and further their children’s education. By providing cash assistance now, the programs reduce 
economic hardship in the short term. By encouraging families to invest in their own develop- 
ment, they hope to reduce poverty over the longer term. In general, the programs have been 
successful at reducing poverty and increasing schooling and health care use. ' 

Before the implementation of Family Rewards, however, the concept had never been 
tested in a high-income country. First launched in 2007 in New York City and sponsored by the 
New York City Center for Economic Opportunity (CEO), Family Rewards offered low-income 
families residing in high-poverty neighborhoods cash incentives (or rewards) for activities 
related to children’s education, family preventive health care, and parents’ employment. The 
program was evaluated using a randomized controlled trial, in which families were randomly 
assigned to a program group that was offered the program or a control group that was not. 

While the program’s effects were more modest than hoped for, it did lead to notable re- 
ductions in poverty in the short term and to increased graduation rates for more academically 
proficient ninth-graders. It had no effect on education outcomes for younger children, however.^ 
Overall, it led to enough positive changes to suggest that it could be made more effective if 
modified somewhat or aimed at certain types of families. The lessons learned from the evalua- 
tion of this first version of Family Rewards (referred to in this report as Family Rewards 1 .0) 
were applied to the next, improved version. 

That next version, referred to here as Family Rewards 2.0, is the subject of this report. 
While still offering rewards in the areas of children’s education, family health, and parents’ 
work. Family Rewards 2.0 is a refinement of the original model in several ways: it offers fewer 
rewards in each domain in order to make the program easier to understand and to focus fami- 
lies’ attention on a limited number of outcomes; it offers the education rewards only to high 
school students, given the lack of effects found for younger students; and it makes the rewards 
more timely, and thus more salient to families, by paying families monthly for rewards earned, 
rather than every two months, and by rewarding students for grades earned. Most notably. 


'Fiszbein and Schady (2009). 

^Riccio et al. (2010); Riccio et al. (2013). 
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however, the new model offers guidance, in which staff members actively engage families in 
conversations about strategies to earn rewards. 

Family Rewards 2.0 was launched in July 2011 in the Bronx, New York and Memphis, 
Tennessee. In each place, about 1,200 families were recmited for the study. Half were randomly 
assigned to a program group, eligible for Family Rewards, and half were assigned to a control 
group, not eligible for the program. Rewards are being offered for three years, and the evalua- 
tion will track these families for several years to determine the effects of the program on 
poverty, children’s education, family health, and parents’ work. 

The evaluation will assess whether the changes to the program model lead to bigger ef- 
fects on behavior in the short and long term. For example, does the new model help more 
families earn rewards, increasing receipt among more disadvantaged families, who earned the 
least from the original program? Does the new model ultimately lead to larger and more broad- 
based effects on school achievement, health, and work? 

While a later report will examine the program’s impacts, this report presents early find- 
ings on its implementation. It discusses how the program was put in place in both cities, the 
challenges posed by the new design, and families’ receipt of rewards through the second 
program year. It considers, for example, whether the new program’s attempt at simplification 
through fewer rewards and easier reporting requirements has increased families’ understanding 
of the program and ability to earn rewards. 

The report also examines the challenges of incorporating active family guidance into a 
conditional cash transfer model. How did staff members engage participants, and were they 
adequately prepared to provide guidance to participating families, many of whom had im- 
portant barriers to earning rewards? Was it practical to train these paraprofessional staff 
members in the more intensive type of advising incorporated into the program in Year 2, called 
motivational interviewing? Motivational interviewing has been shown to be effective at 
eliciting behavioral change, but it is typically practiced by clinicians in health-promotion and 
addiction-treatment settings. 

In brief, the findings indicate that, after some recmitment and start-up challenges in 
Year 1, the program was operating generally as envisioned in both cities by the middle of Year 
2. The family guidance component evolved considerably over time, starting out as fairly modest 
and transactional during Year 1 and becoming more intensive during Year 2. Families earned 
on average more than $2,000 in rewards in Year 2. Finally, the changes to the model seem to 
have led to increased understanding and reward receipt among families. 

Funded by the Social Innovation Fund (SIF) of the Corporation for National and Com- 
munity Service and private organizations. Family Rewards is mn by the Family Rewards 
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Collaborative, which includes members from The Mayor’s Fund to Advance New York City, 
CEO, and MDRC. The Mayor’s Fund to Advance New York City is a grant-making institution 
that facilitates public-private partnerships throughout New York City. CEO, a unit in the New 
York City Mayor’s Office, designs, implements, and evaluates innovative antipoverty pro- 
grams. MDRC worked in close partnership with CEO to design the program and is leading the 
evaluation. MDRC is a nonpartisan social policy research firm with extensive experience 
conducting large-scale projects using random assignment research designs to build rigorous 
evidence on what works to improve the well-being of low-income families. 

The next section reviews the results to date from the original Family Rewards program 
and touches briefly upon findings from conditional cash transfer programs in other countries. 
The remainder of the chapter describes the essential elements of the current program. The 
subsequent sections discuss the two cities involved, the recmitment strategies used, and the 
characteristics of the families enrolled. The chapter ends with an overview of the evaluation and 
the primary research questions. 


Family Rewards 1.0 

Run from 2007 to 2010, the first version of Family Rewards was tested in six of New York 
City’s highest-poverty neighborhoods. The study included approximately 4,800 families and 
11,000 children. The idea for Family Rewards came from the conditional cash transfer pro- 
grams that have become popular in low- and middle-income countries. One of the first such 
programs, and the one that has been the most studied, is Mexico’s Oportunidades program, 
launched as PROGRESA in 1997. There are now conditional cash transfer programs of varying 
scope and scale in most Eatin American countries, as well as in several countries in Africa, the 
Middle East, and Asia.^ 

Like all conditional cash transfer programs. Family Rewards 1 .0 was based on the as- 
sumption that, for a variety of reasons, families may underinvest in their own development even 
though such investments can have long-term benefits.'* Financial incentives can help change 
their calculations, encouraging them to make extra investments of time and energy in certain 
educational, health care, and work-related efforts. Furthermore, the extra resources can help 
make it more feasible for low-income people to undertake such efforts in the short term, by 
helping them pay for educational materials or tutoring for children, for example, or transporta- 
tion to a free dental clinic, or clothes for a job interview. In this sense, the conditional rewards 
may function not only as incentives to action but also as enabling resources. 

^Fiszbein and Schady (2009). 

^^See Riccio et al. (2010) for more detail about the origins of and theory behind conditional cash transfer 
programs and Family Rewards 1.0. 
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The program offered families 22 cash rewards, covering activities and outcomes in 
three domains. In the education area, rewards were offered for parents’ attendance at parent- 
teacher conferences and for children’s attendance in school, performance on standardized tests, 
completion of adequate credits per year, passing of exams required to graduate, and graduation. 
In the health area, rewards were offered for maintaining health coverage and for preventive 
medical and dental checkups for each family member. In the work area, parents were offered 
rewards for sustained full-time work and for pursuing education or training while working.^ 

Part of the strategy behind offering multiple rewards within each domain was to 
help generate a significant effect on income and poverty. The program was designed so that 
a family receiving all or most of the rewards would receive a substantial cash transfer, equal 
to about 25 percent to 30 percent of family income. In addition, 22 rewards meant that 
families had multiple opportunities to earn money. If some rewards remained out of reach, 
others were not. Families received payments every two months, covering all rewards earned 
during a given payment period. 

A final feature of the program is that it offered no services or active counseling. Staff 
members at the neighborhood partner organizations (NPOs), the local nonprofit organizations 
that were the program’s link to participants in the communities, did not develop action plans 
with families to help them earn the rewards, follow up with individual families during the 
program, or provide direct services such as tutoring, test preparation, or job-search assistance, 
although they could refer individuals to these services. Case management and direct services 
were deliberately excluded from the model in order to test the effectiveness of the incentives 
alone. In addition, it was expected that families in New York City would have access to case 
management and services through other programs in the community. The NPOs marketed the 
program aggressively to participants, sponsored various group events and workshops, and, 
when requested to do so, assisted participants in submitting their claims to receive their rewards, 
but they did not manage individual cases or undertake outreach to families who were not 
receiving many rewards. 

During the three years the program operated, the average participating family earned 
nearly $9,000 in rewards, or roughly $3,000 in each year. Over 90 percent of families earned at 
least one education reward and one health reward in each year, while just over 40 percent 
earned at least one work reward. As program designers intended, families were able to and did 
earn rewards across a broad range of areas, and the significant amount earned increased their 
incomes by 22 percent on average and reduced poverty and severe poverty by 12 percentage 


^At the end of the second year, in an effort to simplify the program and reduce its costs, several rewards 
were eliminated, including those for maintaining health coverage and for school attendance for elementary and 
middle school students. 
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points. However, the reductions in poverty and hardship began to diminish after Year 3, when 
the program ended.'’ 

In terms of children’s school progress, Family Rewards improved outcomes for the 
ninth-graders in the study who were performing at a proficient level or better academically 
when they entered, with sizable effects on grade promotion and on graduation. In contrast, the 
program had no effect for lower-performing ninth-graders or for elementary and middle school 
students. In the health area, early, positive effects on visits to the doctor and health status faded, 
in part because families already used preventive care at a fairly high rate, although there were 
continued impacts on health coverage and, especially, dental visits. Finally, the program led to 
modest increases in employment throughout the follow-up period, although much of this new 
work was in jobs not reported to the unemployment insurance system. These jobs are more 
likely to be informal and less likely to offer benefits. Also, the program led to a reduction in 
unemployment insurance-covered work for the more disadvantaged adults in the study. 

The findings add to, and in many ways are consistent with, a growing body of research 
in the United States on the effects of incentives on behavior. In the education domain, for 
example, a number of studies have documented the positive effects of incentives on students’ 
effort.^ More immediate incentives tend to have larger effects than rewards paid in the future, 
for example, and rewards framed as losses can have larger effects than those framed more 
traditionally as bonuses.* Findings from incentive programs in four large school districts also 
suggest that incentives should reward efforts (for example, reading and homework) not results 
(for example, test scores), since students and their parents may not fully understand how to 
improve performance.® However, it is administratively more difficult to reward efforts such as 
reading and homework than it is to reward outcomes such as test scores. Another finding from 
several of these studies, including Family Rewards 1 .0, is that the incentives had effects mostly 
for students on the margins of higher performance (that is, students who were within reach of 
the outcome that was rewarded).'® 

A number of studies document that incentives can affect health-related behaviors such 
as exercise, weight loss, and smoking, although the effects are typically short-lived." Several 
studies also document that incentives can increase employment, although the effects are 
sometimes also short-lived because the employment rates of individuals in the control group 


®See Riccio et al. (2013) for the Ml set of findings. 

^Bettinger (2012); Jackson (2010); Fryer (2011). 

*Levitt, List, Neckermann, and Sadoff (2012); Fryer, Levitt, List, and Sadoff (2012). 
®Fiyer(2011). 

'"Bettinger (2012); Angrist and Lavy (2009); Leuven, Oosterbeek, and van der Klaauw (2010). 
"Volpp et al. (2009); Volpp et al. (2008); Sindelar (2008). 
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tend to “catch up” with those of the program group. Some studies document that incentives 
plus services are more effective at increasing employment than incentives alone, suggesting that 
to take advantage of the rewards, some people may need additional help moving into work. 


Family Rewards 2.0 

In 2010, the Corporation for National and Community Service made its first set of SIF grants 
(see Box 1.1). CEO and the Mayor’s Fund to Advance NYC received a grant to replicate and 
evaluate five of CEO’s most promising antipoverty programs. Family Rewards was one of the 
programs. The replication effort makes several important modifications to the original Family 
Rewards model, based on the experience of and findings from the first evaluation. In addition, 
the replication was designed to answer the question of whether the model would work outside 


Box 1.1 

The Social Innovation Fund 

The SIF, a key White House initiative and program of the Corporation for National and 
Community Service, combines public and private resources to grow the impact of innova- 
tive, community-based solutions that have compelling evidence of improving the lives of 
people in low-income communities throughout the United States. The SIF invests in three 
priority areas: economic opportunity, healthy futures, and youth development. 

The SIF makes grants to experienced grant-making “intermediaries” that are well posi- 
tioned within communities to identify the most promising programs and guide them 
toward greater impact and stronger evidence of success. These grants range from $1 
million to $10 million annually for up to five years. The intermediaries then match the 
federal funds dollar for dollar, and hold open competitions to identify high-performing 
nonprofit organizations working in low-income communities that have innovative solu- 
tions with evidence of compelling results. Once selected, these nonprofits must also match 
the funds they receive, and participate in rigorous evaluations of the impact of their 
programs. The nonprofits share data, lessons learned, and results, helping to build the 
capabilities of the social sector and lifting up solutions that can transform lives and com- 
munities. 


*^Holt (2006); Eissa and Hoynes (2006); Michalopoulos (2005); Hendra et al. (2011); Martinson and Hen- 
dra (2006). 

'^Michalopoulos (2005). 
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New York City by testing it in New York City and in Memphis, Tennessee. The Bronx was 
selected as the area within New York City to be included in this evaluation because it contains a 
large number of high-poverty neighborhoods. ''' Memphis was selected as a second site because 
its mayor was interested in trying bold new approaches to reducing poverty. Memphis offered 
the opportunity to test the Family Rewards model in a city with different economic and social 
conditions from New York and in a state with a different social safety net. 

Program Delivery 

The demonstration is being mn by the Family Rewards Collaborative, which includes 
The Mayor’s Fund to Advance New York City, CEO, and MDRC (Figure 1.1). The Family 
Rewards Collaborative sets broader design policies and provides technical assistance oversight, 
largely through the lead program operator. Seedco, a workforce and economic development 
organization, was selected to lead the operation of the program based on its expertise and 
experience managing the first Family Rewards. Seedco works with local organizations to create 
economic opportunities for disadvantaged individuals. The lead program operator’s roles in 
Family Rewards 2.0 are to manage the contracts with the NPOs, and to provide them technical 
assistance on program implementation by developing management information systems, 
program tools, training materials, and performance management reports. It also mns the 
payment-processing system for both cities. 

Seedco selected four NPOs to operate the program. The NPOs were selected based on 
the experience they had in program delivery in their local communities. In the Bronx, the NPOs 
were the Children’s Aid Society and BronxWorks. The Children’s Aid Society (CAS), founded 
in New York City in 1853, provides a range of services to children and parents, including early 
education, housing assistance, adult education services, benefit assistance, and legal assistance. 
BronxWorks has served the Bronx for over 40 years. Originally named the Citizens Advice 
Bureau, BronxWorks offers a wide range of services and programs for Bronx residents, includ- 
ing services related to workforce development, benefit assistance, food assistance, housing 
assistance, education, and child care. 

In Memphis, the NPOs were Urban Strategies Memphis HOPE, referred to hereafter as 
Memphis HOPE, and Porter-Eeath. Memphis HOPE was created in 2004 by the Women’s 
Foundation for a Greater Memphis, in collaboration with the City of Memphis and the Memphis 
Housing Authority. The organization provides comprehensive services for low-income women 
and their children in targeted communities. Memphis HOPE works with over 40 community 
partners to provide services in the areas of youth development, case management, housing 


'“'The Bronx also participated in the original evaluation, although that evaluation targeted different neigh- 
borhoods. 
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SIF Family Rewards 
Figure 1.1 

Organizational Structure 




assistance, education, benefits, and health. Originally started to serve widows and orphans in the 
1850s, Porter-Leath currently offers services to more than 10,000 low-income parents and 
children in the Memphis area. Services include early education, foster care, benefits assistance, 
and parenting guidance. 

All NPOs are charged with implementing core components of the program such as re- 
cmitment, outreach and family guidance, and the Family Earning Plan. Each NPO has its own 
program supervisor, who is responsible for multiple programs at the NPO, including Family 
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Rewards. The supervisor directly manages three Advisors for Family Rewards, which allows 
each NPO some level of autonomy in delivering the program elements. 

In summer 2012, CAS was elevated to the role of lead program operator, replacing 
Seedco.'^ CAS was a natural candidate to lead the program, as it was deeply familiar with 
Family Rewards and had experience managing multisite and federally funded projects. To 
ensure program continuity. Family Rewards staff members at Seedco were transferred to CAS. 
The contract transition went smoothly, without dismption to services for families. To avoid 
confusion between CAS’ role leading operations and its role as an NPO, the text will hereafter 
distinguish between these roles with the terms “CAS-Central” and “CAS-NPO.” 

The Rewards 

Table 1.1 presents the rewards for Family Rewards 2.0. The top panel shows the educa- 
tion domain, in which students are rewarded for high attendance, good grades, performance on 
state core exams, and the taking of college entrance exams. One addition to the reward list 
under the new program is a reward for grades, in an effort to make the rewards more timely and 
more strongly tied to daily performance. The second panel shows rewards for health, in which 
families receive payments for medical and dental checkups for each family member. The new 
program removes the reward for maintaining health insurance, since most families in the 
original study had coverage. Finally, the program rewards parents for maintaining full-time 
work and earning a General Educational Development (GED) certificate. The original model 
rewarded a range of approved training courses, but few families received the training reward. 
For this reason, program designers decided to focus on one outcome, the GED certificate, which 
is well known to participants and can serve as a stepping-stone to further training. (See Appen- 
dix Table A.l for a side-by-side comparison of specific rewards for the two programs.) 

As in Family Rewards 1.0, families receive payments for meeting reward milestones in 
one of two ways. Two of the rewards, for attendance and for state core exams, are automatically 
verified using school records, requiring no action on the part of the family to earn the payment. 
All other rewards must be claimed by submitting a “coupon” to the program, verifying that the 
milestone has been met. An example is the reward for full-time work, which requires coupon 
submission along with other items of verification. Coupons are paper documents listing the 
name of the activity, information about the participant, the dollar value for completing the 
activity, and instructions for submission. At the start of each program year, families pick up 
coupon books containing all of the coupons for which they might be eligible. 


'^Seedco assigned the SIF projects to CAS after determining that organizational issues impeded its ability 
to manage the work effectively. In 2012, investigations by local and federal officials uncovered irregularities in 
Seedco’s administration of workforce services unrelated to Family Rewards. 
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SIF Family Rewards 
Table 1.1 


Family Rewards 2,0: Schedule of Rewards 


Domain 

Bronx 

Memphis 

Education incentives 

High school students only 



Attends 95% of scheduled 
school days 

$40 per month 

$40 per month 

Takes an SAT or ACT exam 
(once during program) 

$50 

$50; must score 19 or more 
on the ACT if administered 
by Memphis City Schools'* 

Receives grades on an official 
report card*” 

$30 per A (90-100); $20 per 
B (80-89); $10 per C (75-79) 

$30 per A (93-100); $20 per 
B (85-92); $10 per C (75-84) 

Passes up to 5 Regents exams'^ 
or 7 End of Course exams'^ 

$500 per Regents exam for a 
score of 75 or above; $400 
per exam for a score of 65-74 

$200 per End of Course exam 
for a score of proficient or 
advanced (increased to $300 
in Year 2) 

Health incentives 

Parents and children 19 and younge 

r 


Annual medical checkup 

$100 per family member 

$100 per family member 

Preventive dental care every 
6 months (once per year for 
children 1-5 years old) 

$100 per family member per 
visit 

$100 per family member per 
visit 

Workforce incentives 

Parents only 



Sustains full-time employment 

$150 per month 

$150 per month 

Earns a GED certificate 

$400 

$400 


(continued) 
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Table 1.1 (continued) 


SOURCE: CAS's Family Rewards program materials. 

NOTES: GED = General Educational Development. 

“The ACT is scored out of 36. Memphis City Schools officials requested that the minimum score for 
the reward be set at 19 for students taking the test for free in class because this is considered an indication 
that students are ready for college-level work. 

'’In Year 1, the amount of each reward for grades was prorated based on the number of official report 
cards issued by a student's school. To simplify verification in Year 2, students were paid the listed 
amounts for their grades regardless of the number of official report cards they received, up to a maximum 
of $600 per program year. 

“High school students (grades 9-12) in the Bronx were eligible to earn rewards for the following Regents 
exams: English, one of any math exams (including Math A, Math B, Integrated Algebra, Geometry, and 
Algebra 2/Trigonometry), U.S. History and Government, Global History and Geography, and one of any 
science exams (including Living Environment, Chemistry, Physics, and Earth Science). 

“^High school students (grades 9-12) in Memphis were eligible to earn rewards for the following End of 
Course exams: Algebra 1, Algebra 2, Biology, English 1, English 2, English 3, and U.S. History. 


In an effort to make the rewards more timely and thus more salient, Family Rewards 2.0 
makes it possible for families to receive payments for rewards earned every month, along with 
an Earnings Statement that documents a family’s earnings and any rewards that the family 
submitted coupons for but did not earn. Finally, families were strongly encouraged to open low- 
fee bank accounts in order to take advantage of the direct deposit of rewards. Many families did 
not have bank accounts when they entered the study and opted to open these new accounts. 

Family Guidance 

Family Rewards 2.0 includes an active guidance role for the staff, given the earlier find- 
ings suggesting that families might have benefited from such guidance. Under the new model, 
staff members at the NPOs develop a Family Earning Plan with every family and meet with 
them twice per year to discuss their progress toward earning the rewards. Staff members reach 
out to engage families who are not earning rewards and have access to a small “resource fund.” 
Families can receive nonreward money from this fund at the staffs discretion to help them 
invest in services they might need to help earn rewards, such as short-term tutoring, work 
uniforms, transportation to job interviews, or licensing fees. As discussed in more detail in 
Chapter 3, the strategy for family guidance shifted during Year 2, as staff members were trained 
in the use of motivational interviewing. Motivational interviewing, a client-driven counseling 
method, has been found in other settings to be effective at eliciting behavioral change. 
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Target Groups 

The program targeted families with at least one child entering ninth grade or tenth 
grade. Once enrolled in Family Rewards 2.0, however, all of the family’s children under age 20 
became eligible for the health-related rewards. The program also targeted families receiving 
Temporary Assistance for Needy Families (TANF) and Supplemental Nutritional Assistance 
Program (food stamp) benefits, in order to direct resources to the neediest families and to 
explore how a conditional cash transfer approach might be integrated into these safety-net 
programs. 


The Cities 

In some respects, Memphis represents an ideal place for the replication effort. Although both 
areas have relatively high poverty rates (29 percent of Bronx residents have incomes below the 
poverty line compared with 26 percent of Memphis residents), Memphis provides quite a 
different context from New York City. Memphis has a much smaller population than New York 
(roughly 700,000 people in Memphis compared with more than 8 million in New York as a 
whole, and 1.4 million in the Bronx). Memphis’ population is largely African American (63 
percent) or white, non-Hispanic, compared with a population in the Bronx that is majority 
Hispanic or African American. A higher proportion of Memphis residents are high school 
graduates than in the Bronx, and median household income is somewhat higher ($36,817 in 
Memphis compared with $34,300 in the Bronx).'*’ 

Local labor market data are available for Memphis and the greater New York metropol- 
itan area, rather than the Bronx specifically. Although the unemployment rate was similar in 
both areas in early 2013,'^ median wages were lower in Memphis (about $15 per hour) than the 
New York area ($21 per hour).'* Although this difference reflects in part the different mix of 
occupations in the two areas, median wages are also lower in Memphis for specific occupations, 
such as for “home health aide” jobs or for jobs in “food preparation and service.” 

There are also some differences between the cities in the types of benefits available to 
residents. Through state policy, both cities have a fairly wide array of benefit programs availa- 
ble, including TANF, food stamps, and Medicaid.'^ However, New York residents have access 
to a somewhat larger number of programs, and the benefits available are more generous. For 
example, TANF benefits are higher in New York than in Tennessee,^" and the state-sponsored 


'^All figures in this paragraph are from U.S. Census Bureau (2014). 
'^U.S. Department of Labor, Bureau of Labor Statistics (2014a). 
'*U.S. Department of Labor, Bureau of Labor Statistics (2014b). 
'"Benefits.gov (2014). 

^"Kassabian, Huber, Cohen, and Giannarelli (2013). 
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health insurance program for low-income New Yorkers covers routine dental care, while the 
similar plan in Memphis (TennCare) does not. Dental care coverage, in particular, may affect 
the ability to cam the dental reward. 

Two additional factors that might affect the implementation of the program and how 
families respond to it are related to the cities’ school and transportation systems. The Memphis 
school system, covering all of the high schools attended by students in the study, is much more 
centralized than the school system in New York. While students in Memphis can attend over 
50 high schools. New York students can attend over 400 high schools across the city, and New 
York City Department of Education policy allows schools substantial discretion in setting 
policies related to report cards, schedules, and attendance. As discussed in a later chapter, these 
differences created some challenges in designing the education rewards across the two cities. 
Meanwhile, families and students in the Bronx have access to an extensive public transporta- 
tion system, whereas families in Memphis rely heavily on cars. The lack of good public 
transportation made study recmitment more difficult and also affected how families participat- 
ed in the program. 

The two cities also have different numbers and types of community-based organiza- 
tions, which affects the resources families have available to help them earn various rewards. At 
the outset of the program, each NPO created a “resource guide” to assist its staff in referring 
participants to various services. Although these guides are unlikely to represent all of the 
resources available to families in a community, they can provide a rough picture of the service 
environment facing participating families, or at minimum the types of services to which 
participants might be directed by staff members. A comparison of the guides of the four NPOs 
reveals some differences between cities. In the education domain, the New York resource 
guides more explicitly target services to improve academic achievement, while the Memphis 
guides list more comprehensive services, such as classes to teach parents about their children’s 
educational needs and problem-solving skills, and evaluations to diagnose learning disabilities. 
Many of the resources in the New York guides are also web-based (for example, tutoring, test 
preparation, etc.). The New York guides have twice as many workforce services listed as the 
Memphis guides. Finally, a notable difference between the two cities is that most of the organi- 
zations and resources listed in the New York guides are located in the Bronx or online, a 
deliberate decision made by the Bronx NPOs, while most of the resources in the Memphis 
guides are in the downtown or midtown area, areas that are not close to where participants live. 
Since transportation is a challenge for many Memphis residents, this may keep them from 
having access to services. Thus, overall, Bronx residents may have easier access to resources 
that might help them earn rewards in all three domains. 
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Recruitment and the Sample 

The NPOs were charged with recruiting 1 ,200 families in each city. Families were eligible for 
the study if they met the following criteria: 

1. They received TANF or food stamp benefits. 

2. They lived in one of five targeted community districts in the Bronx or anywhere in 
the city of Memphis. 

3. They had an adult at least 1 8 years of age. 

4. That adult was a U.S. citizen or U.S. legal permanent resident. 

5. The household included at least one child entering ninth or tenth grade in the 2011 
school year. 

Recmitment began in August 2011 in the Bronx and September 2011 in Memphis, 
when the NPOs received lists of potentially eligible participants provided by the human services 
agencies and departments of education in each city.^' NPO staff members were then required to 
call families on the lists, making multiple attempts to reach eligible families, including addition- 
al phone calls and home visits. 

The pace of recmitment varied in each city, as staff members faced challenges to reach- 
ing potential participants, including outdated or incorrect contact information, a high level of 
skepticism among potential participants about the program, concern among some potential 
participants about the effect participation might have on benefits they were receiving, and lack 
of transportation (particularly in Memphis). 

Ultimately, NPOs resorted to a range of recmitment strategies in order to reach their 
targets. For example, in the Bronx, an additional letter was sent to potentially eligible families in 

^’in New York, the recruitment list was provided by the Human Resources Administration. Families were 
chosen for recruitment if they received TANF or food stamps and had children between the ages of 14 and 16. 
Only those families who were receiving TANF or food stamps at the time of enrollment were randomly 
assigned into the study. Grade-level data from the Department of Education were matched with the recruitment 
list to help make a priority of families most likely to have children entering the ninth or tenth grade. In 
Memphis, the recruitment list was provided by Memphis City Schools. Families were chosen for recruitment if 
they had children entering the ninth or tenth grade. These data did not include information about benefit status, 
so enrollees in Memphis were determined to be eligible for Family Rewards based on their own reports of 
receiving TANF or food stamps. 

^^The cash transfers do not affect eligibility or payment amounts for most existing government transfer 
benefits, including TANF, food stamps, Medicaid, the Children’s Health Insurance Program, housing assis- 
tance, or the Earned Income Tax Credit. 
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November, and information about the study was added to the city’s public-service call center. In 
Memphis, the local press and the mayor’s office promoted the study, the NPOs set up satellite 
intake offices at two local libraries, and they began accepting walk-ins, or eligible families who 
were not on the NPOs’ existing lists but who had heard about the program through local media. 

The pace of enrollment meant that the full launch of the program was delayed by 
several months. While enrollment was originally expected to conclude by the end of 
September 2011, in practice the majority of families in the study enrolled after that point, 
and many enrolled as late as January or February 2012. For this reason, this report focuses 
on reward receipt during the second program year, when all families were eligible to earn a 
full year of rewards. 

In total, Bronx NPOs enrolled 1,232 households and Memphis NPOs enrolled 1,226 
households. Tables 1.2, 1.3, and 1.4 present selected characteristics of the samples. There are 
some differences between the cities. As shown in Table 1.2, for example, a larger fraction of 
participating families are headed by a single parent in Memphis (90 percent) than in the Bronx 
(77 percent). A much larger fraction of the sample in the Bronx than Memphis is Hispanic and 
speaks Spanish as a primary language. In contrast, nearly all participants in Memphis are 
African American. Probably reflecting the tighter housing market in New York, more families 
in the Bronx lived in subsidized housing when they entered the study: 35 percent of the Bronx 
families reported living in public housing, compared with only 10 percent of Memphis families. 
In contrast, the number of families receiving TANF benefits is much higher in Memphis, at 3 1 
percent, than in the Bronx, at 5 percent. The last rows of Table 1.2 illustrate the material 
hardships faced by enrolled families. More than half of families reported being unable to pay a 
rent or utility bill within the previous year, and a quarter were unable to pay for food or needed 
medicine. 

Table 1.3 shows that adults in the Bronx had somewhat lower education levels when 
they enrolled than those in Memphis, in part reflecting the fact that more adults in the Bronx are 
immigrants. (Foreign-bom study participants in the Bronx had lower education levels than their 
counterparts bom in the United States.) For example, nearly half of the Bronx sample adults did 
not have either a GED certificate or a high school diploma at the time of study enrollment, 
compared with only 3 1 percent of adults in Memphis. In contrast, adults in the Bronx sample 
were more likely to be working at the time of study entry (57 percent) than their counterparts in 
Memphis (44 percent). This difference is also related to immigrant status, given that foreign- 
bom study participants in the New York sample were more likely to be working than their 
counterparts bom in the United States. 
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SIF Family Rewards 
Table 1.2 

Selected Household Characteristics at Enrollment, by City 


Characteristic 

Bronx 

Memphis 

Demographic characteristics 

One-parent family^ (%) 

77.1 

90.4 

Family with two adults enrolled in Family Rewards (%) 

6.8 

2.5 

Current marital status (%) 

Single 

49.8 

70.8 

Married 

20.9 

8.0 

Separated 

18.7 

13.2 

Divorced 

7.4 

5.0 

Living with partner or legal domestic partner, or widow/widower 

3.2 

3.0 

Average number of children in household (under 19 years old) 

2.4 

3.0 

Average number of high school students in household (under 19 years old) 

1.3 

1.4 

Primary language (%) 

English 

48.2 

99.1 

Spanish 

49.8 

0.6 

Other 

2.0 

0.3 

Household characteristics (%) 

Currently receive food stamps 

99.4 

98.0 

Currently receive TANF or other safety-net assistance 

5.0 

30.5 

Live in rental housing 

94.3 

71.6 

Live in public housing 

34.7 

10.3 

Receive Section 8 rental assistance 

23.5 

13.6 

During the last 12 months, ever unable to pay: 

Rent or utility bills 

49.2 

64.0 

Telephone bills 

30.4 

32.8 

Food or prescription drug costs 

25.1 

24.2 

Sample size 

1,232 

1,226 


SOURCES: MDRC calculations from baseline survey and random assignment module data. 

NOTES: Percentages for some categories may not add up to 100 due to rounding or missing values. 

^This measure includes families with parents who reported their marital status as single, single but 
living with a boyfriend or girlfriend, separated, divorced, or widowed. 
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SIF Family Rewards 
Table 1.3 

Selected Adult Characteristics at Enrollment, by City 


Characteristic 

Bronx 

Memphis 

Demographic characteristics 

Female (%) 

87.7 

94.0 

Average age (years) 

42.6 

39.3 

Race/ethnicity (%) 

Hispanic 

74.4 

0.9 

African American 

22.9 

98.1 

Other 

2.7 

1.0 

Citizenship status (%) 

U.S. citizen by birth 

42.7 

98.6 

U.S. citizen by naturalization 

31.8 

0.8 

Legal permanent resident 

25.5 

0.6 

Educational characteristics (%) 

Highest degree received 

GED certificate 

6.2 

7.6 

High school diploma 

32.8 

51.2 

Technical/associate's/2-year college degree 

7.2 

6.2 

4-year college degree or higher 

5.9 

4.0 

None of the above 

48.0 

31.0 

Trade license or certificate 

44.9 

27.6 

Employment characteristics 

Currently employed (%) 

57.1 

44.1 

Full time (at least 30 hours per week) 

39.3 

34.0 

Part time 

17.6 

9.9 

Among adults currently employed 

Average weekly wage ($) 

313 

332 

Has a physical problem that limits work (%) 

16.1 

18.3 

Has an emotional or mental health problem that limits work (%) 

6.5 

7.9 

Receives SSI or SSDI due to disability (%) 

13.1 

15.1 


(continued) 
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Table 1,3 (continued) 


Characteristic 

Bronx 

Memphis 

Medical coverage (%) 

Source of health insurance coverage 

Public health insurance 

85.9 

81.9 

Employer-provided or other health insurance 

10.3 

10.1 

No coverage 

3.8 

8.0 

Medical and dental care (%) 

Usual source of medical care 

Clinic or health center 

73.1 

49.7 

Doctor's office or health maintenance organization 

16.2 

41.0 

Hospital emergency room 

3.3 

7.8 

Hospital outpatient department 

7.2 

1.0 

Other 

0.2 

0.5 

Date of last medical checkup 

Less than 6 months ago 

57.7 

39.1 

6 to 12 months ago 

29.7 

37.4 

More than 12 months ago or never had a checkup 

12.6 

23.5 

Date of last dental checkup 

Less than 6 months ago 

45.7 

13.0 

6 to 12 months ago 

32.2 

22.7 

More than 12 months ago or never had a checkup 

22.1 

64.3 

Health status (%) 

General health status 

Excellent/very good 

35.4 

35.8 

Good 

37.9 

38.4 

Fair/poor 

26.7 

25.8 

Felt down, depressed, or hopeless during the past 2 weeks 

22.2 

23.3 

Financial characteristics (%) 

Has a bank or credit union account 

59.2 

39.7 

Sample size 

1,314 

1,256 


SOURCES: MDRC calculations from baseline survey and random assignment module data. 

NOTES: Percentages for some categories may not add up to 100 due to rounding or missing values. 
Italics indicate measures calculated for a subset of the full sample. 

SSI = Supplemental Security Income. SSDI = Social Security Disability Insurance. 
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Finally, Table 1.4 presents selected characteristics of the children in participating fami- 
lies, including the target ninth- and tenth-graders and their siblings. Other than race and ethnici- 
ty and English Language Learner status, there are no large differences between the cities in the 
samples of children. Most children had health coverage at study entry, about 15 percent had 
received an Individualized Education Program (lEP),^^ and most parents reported that their 
children were in excellent or good health. 


The Evaluation 

The evaluation includes a comprehensive study of Family Rewards 2.0 and is stmctured around 
the following three components: 

Implementation Analysis 

The implementation analysis is the focus of this report. It will explore the operations 
and implementation of Family Rewards 2.0 and families’ participation in it. It will examine the 
stmcture of the program, including the roles and responsibilities of each of the partners. This 
analysis will include a description and assessment of how families were brought into the study 
and how the conditional cash transfer offer was marketed and communicated to families. The 
analysis will also examine the implementation of the family guidance component of the model 
and how it may have affected participants’ experiences of the program. Data for the implemen- 
tation analysis include observations of program activities (including family guidance sessions), 
interviews with Advisors at the NPOs and CAS-Central, focus groups with 85 adults and 57 
high school students, a review of all program materials, a case-file review to analyze the 
implementation of the Family Laming Plans, and an analysis of the management information 
system and payment-processing data. 

Impact Analysis 

The impact analysis will examine the effects of the program on families participating in 
the study. Because families were assigned at random to either the program or control group, any 
differences in outcomes can be attributed to the program with confidence. The study will track 
families in both the program and control groups for three years using administrative records and 


^^lEPs are mandated by the Individuals with Disabilities Education Act, to help children with disabilities 
achieve their educational goals. The percentage of students with an lEP is similar for the high school students 
in the sample and for their younger siblings. 
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SIF Family Rewards 
Table 1.4 


Selected Child Characteristics at Enrollment, by City 


Characteristic 

Bronx 

Memphis 

Demographic characteristics 



Female (%) 

50.4 

51.6 

Age (%) 



0-5 years 

10.8 

9.6 

6-10 years 

16.2 

18.4 

11-13 years 

14.8 

18.4 

14 years or older 

58.2 

53.6 

Average age (years) 

12.4 

12.3 

Race/ ethnicity (%) 



Hispanic 

72.6 

1.0 

African American 

23.7 

97.7 

Other 

3.7 

1.4 

Bom in the United States (includes Puerto Rico, Guam, 
U.S. Virgin Islands, and Northern Marianas) (%) 

87.3 

99.6 

Educational characteristics (%) 



Expected grade level in September 2011 



Not yet in grade 1 

10.1 

9.7 

Grades 1-5 

17.3 

19.7 

Grades 6-8 

13.1 

18.6 

Grade 9 

25.0 

22.0 

Grade 10 

21.0 

18.9 

Grades 11-12 

8.3 

9.5 

In college 

3.8 

0.8 

Not enrolled 

1.4 

0.9 

Type of school attended in the previous school year 



Public or charter 

91.2 

85.4 

Private or parochial 

0.7 

6.9 

College 

2.9 

1.6 

Not enrolled 

5.3 

6.1 

Enrolled as an English Language Learner in the previous school year 

21.6 

5.5 

Had an Individualized Education Program in the previous school year 

16.3 

13.8 

Received tutoring help in the previous school year 

26.5 

30.0 

Nonproficient on English exarn^ 

74.8 

85.6 


(continued) 


20 






Table 1,4 (continued) 


Characteristic 

Bronx 

Memphis 

Medical coverage (%) 

Source of health insurance coverage 

Public health insurance 

90.3 

95.7 

Employer-provided or other health insurance 

8.0 

3.8 

No coverage 

1.7 

0.5 

Medical and dental care (%) 

Usual source of medical care 

Clinic or health center 

79.8 

55.3 

Doctor's office or health maintenance organization 

14.3 

41.3 

Other 

5.9 

3.4 

Date of last medical checkup 

Less than 6 months ago 

64.3 

42.0 

6 to 12 months ago 

30.0 

46.1 

More than 12 months ago or never had a checkup 

5.7 

11.8 

Date of last dental checkup 

Less than 6 months ago 

51.8 

42.7 

6 to 12 months ago 

34.0 

43.7 

More than 12 months ago or never had a checkup 

14.2 

13.7 

Health status (%) 

General health status 

Excellent/very good 

73.5 

77.4 

Good 

23.6 

19.9 

Fair/poor 

3.0 

2.8 

Has a physical problem that limits activities 

4.7 

5.4 

Has an emotional or mental health problem that limits activities 

6.0 

5.5 

Sample size 

2,920 

3,407 


SOURCES: MDRC calculations from baseline survey, random assignment module, and 
educational records data from the New York City Department of Education and Memphis City 
Schools. 

NOTES: Percentages for some categories may not add up to 100 due to rounding or missing 
values. 

‘‘Proficiency level is only reported for high school students who had taken a standardized test 
to determine proficiency within the two years prior to enrollment. Data were available for most 
students who were in ninth or tenth grade at the time of enrollment. 
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survey data.^"^ The impact analysis will examine the program’s effects on a wide range of 
outcomes related to children’s school performance, family health, parents’ work and earnings, 
and family income and well-being. Data sources to be used include administrative records on 
students’ school performance, parents’ employment and earnings, and benefit receipt. A survey 
was administered to families 24 months after study entry, capturing program experiences, 
income and material hardship, and other outcomes. 

Cost Analysis 

Finally, this analysis will estimate the cost to administer rewards and the cost of reward 
payments to participants. Costs will be estimated for each reward type. Cost data will come 
from the budgets and expenditure reports of program operators as well as reward payment 
records. This information will highlight how much money was transferred to participants in the 
form of rewards as well as how much it cost to administer the transfers. 


Organization of the Report 

This report covers the implementation of Family Rewards 2.0 through the first two years of 
program operations, or from September 2011 through August 2013. Chapter 2 discusses the 
design of the rewards and the rate at which participating families earned each reward. Chapter 3 
covers the family guidance component of the program and how it evolved over the first two 
program years. Chapter 4 compares the rates at which families received rewards in Family 
Rewards 1 .0 and Family Rewards 2.0. As noted earlier, because of the delay in recmiting the 
sample, many families did not have the opportunity to earn a full two years of rewards during 
this time. Thus, all analyses of reward receipt focus on the second program year, when all 
families had the same 12 months of eligibility to earn rewards. Finally, this report offers some 
concluding thoughts and looks ahead to coming reports. 


additional funding is obtained, the evaluation will track study participants and assess impacts for an- 
other two years after the rewards end. 
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Chapter 2 

Delivering and Earning Rewards 


Introduction 

As shown in the program’s theory of change model (Appendix Figure B.l), the centerpiece of 
the Family Rewards intervention is the offer of financial rewards. The rewards were designed to 
promote a set of behaviors that can help low-income families break the cycle of intergeneration- 
al poverty. All of the rewards except the one for grades were offered in Family Rewards 1 .0. ' 
The earlier program gave program designers and the lead program operator an opportunity to 
fine-tune their procedures and taught them valuable lessons about creating financial incentives, 
in particular, the importance of having a short list of rewards. 

Despite this experience, the program operators faced two challenges at the start of the 
second program. The first was replicating and delivering the rewards in a new and very different 
city (see Chapter 1 for a comparison of Memphis and the Bronx). Because the rewards were the 
same in both places, the program designers had to make sure that every incentive was under- 
standable to all participants, meaningful in both cities for achieving the program’s poverty- 
reducing goals, and possible to achieve (and verify) given local conditions and institutions. In 
light of these concerns, one of the main implementation questions is whether Memphis partici- 
pants faced any disadvantage in earning the rewards. Memphis is not as rich in education, 
health, and workforce services as the Bronx. If Memphis participants could not find providers to 
help them meet the conditions for rewards, did they miss opportunities to earn money from the 
program? 

The second challenge program implementers faced was striking the right balance be- 
tween minimizing the burden associated with verifying completion of activities and ensuring 
that participants only received payments they had earned. This is an issue for any financial 
incentives program in which the participants must show they have met the conditions for 
payment. Both the incentives and the verification procedures had to be simple enough to be 
managed by economically distressed families facing multiple life challenges. The short list of 
rewards was designed to be easy to understand, but participants had to submit paperwork 


*The rewards for taking the SAT/ACT and receiving a General Educational Development (GED) certifi- 
cate were defined differently (see Appendix Table A.l). 
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(coupons) to verify six out of the eight rewarded activities, which could have created an 
additional barrier to earning rewards.^ 

This chapter will examine how much money participants earned from the program 
overall and from each type of reward. It will explore staff efforts to operate an accurate and 
efficient reward payment system. In addition, it will describe the burdens or “costs” in time or 
effort the payment system imposed on families, and consider whether there were better ways to 
balance program and participant needs. 


Key Findings 

• Families were highly engaged in the program in Year 2. Ninety-six percent 
of families earned at least one reward that year, and those families earned an 
average of $2,160 of extra household income. Most families (93 percent) 
earned at least one education reward and 79 percent earned a health reward, 
but only 44 percent of families earned at least one workforce reward. Be- 
cause most families did not earn a workforce reward, the education rewards 
earned by high school students typically made up more than half of the mon- 
ey families earned from Family Rewards. 

• A similar percentage of families in Memphis earned the rewards as did in the 
Bronx, but they earned slightly less money on average. This is because fewer 
students in Memphis earned the high-value reward for state core exams, and 
fewer adults claimed the reward for dental checkups. 

• Qualitative data suggest that families had a good understanding of the re- 
wards and what they had to do to get paid for them. This was tme even in the 
work domain, where most adults who submitted a coupon for full-time work 
had at least one rejected. 

• The Children’s Aid Society (CAS)-Central successfully delivered the pay- 
ments owed to families, but because it used a manual system to process 
payments and create Earnings Statements, these payments were not issued as 
soon after activities were completed as the program designers had hoped. 


^The remaining two activities, for attendance and passing state core exams, were verified using adminis- 
trative records that the payment-processing staff requested directly from the two school systems. 
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The Payment-Processing System 

The payment-processing system is a central mechanism in a conditional cash transfer program. 
Participants must receive accurate payments reasonably soon after they are earned and be able 
to connect the payments to their prior actions in order for the rewards to reinforce particular 
behaviors and achieve the program’s short-term goal of transferring cash to families.^ In the 
case of this program, the system must function as effectively for Memphis participants as it does 
for those in the Bronx. 

Many of the procedures required to administer the payment-processing system were put 
in place in Family Rewards 1.0. Seedco, which led operations for both programs until the 
transfer of responsibility to CAS -Central, brought with it a wealth of operational experience. To 
reduce costs and ensure consistency across the two cities, there was one “back office” in 
Manhattan for all participants. This office housed eight payment-processing staff members and 
two supervisors who received documents from participants and administrative records from the 
school districts to verify completion of activities, determined whether participants were eligible 
for payments, and posted payments to their accounts. Two of the bilingual staff members had 
the dual role of responding to questions from participants and Advisors on a Helpline. The 
Helpline had a toll-free number, and was open from 9 a.m. to 5 p.m. Eastern Time. Call volume 
was heaviest during the lunch hour, which was manageable because New York City and 
Memphis are in different time zones. 

Every year, each family received a binder called the “coupon book.” It was the fami- 
ly’s take-home guide to the program. The front section included a summary of the rewarded 
activities, a calendar, and other tip sheets to help families remember deadlines and stay 
organized (Figure 2.1). The rest of the binder contained personalized coupons for each enrolled 
family member. Coupons were double-sided, colorfully designed sheets of paper that listed the 
name of the activity, the name of the family member eligible to use it, the dollar value for 
completing the activity, and instructions for submission. Health coupons had to be filled out by 
a doctor or dentist to prove that the activity was completed. For other coupons, the participant 
had to attach the correct form of additional documentation (for example a report card or pay 
stub) to verify completion. 

Families were paid every month for activities completed about six weeks earlier. The 
lag between completing the activity and receiving payment was due to the time it took to 
manually process payments. For example, the first month of each program year was September. 
Families could qualify for rewards by completing activities from September 1 to 30. Using self- 
addressed, stamped envelopes provided by the program, families were required to collect and 


^Riccio (2010), pp. 92-94. 
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SIF Family Rewards 
Figure 2.1 

Tips and Deadlines from the Year 2 Coupon Book 


February 2013 


Fri 1 


Sat 2 


Sun 3 


Mon 4 


Tues 5 


Wed 6 


Thurs 7 

Registration Deadline for March 9th 
SAT 

Fri 8 

Sat 9 


Sun 10 


Mon 11 


Tues 12 


Wed 13 


Thurs 14 


Fri 15 

Reward Day! 

Sat 16 


Sun 17 


Mon 18 


Tues 19 


We 20 


Thurs 21 


Fri 22 


Sat 23 


Sun 24 


Mon 25 


Tues 26 


Wed 27 


Thurs 28 

Last Day to Visit a Dentist for 
This Year’s First Dental Coupon 



SOURCE: CAS's Family Rewards program materials. 
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mail in their paperwork by October IS/ Around this time, CAS-Cenbal would also request 
adminisbative data to verify education rewards that did not require coupons. Verification tasks, 
including quality checks, took two to three weeks, and payments were posted to accounts on 
November 15. The payment-processing staff created Earnings Statements (Figure 2.2) for each 
family that explained what they had (and had not) been paid for, and translated these for 
Spanish speakers. This process was also manual, so Earnings Statements were received three to 
four weeks after payments. This lag was longer than program designers had hoped for, but it is 
likely to be impossible to improve in the context of a demonstration project that is creating all of 
its own systems to perform verification with a limited number of staff members. 

The Earnings Statement informed participants when their coupons had been rejected 
due to incomplete or incorrect documentation, or when a person claiming a reward was not 
eligible for it. The payment-processing staff tried to avoid rejections by correcting problems 
with coupon submissions themselves or accepting alternative forms of proof that met the 
same standards. For example, the payment-processing staff accepted the medical checkup 
coupon if it included a doctor’s office stamp or doctor’s license number, even if it was 
missing the doctor’s signature. Participants with rejections were encouraged to call their 
Advisors or the Helpline. Memphis participants relied more heavily on the Helpline than 
participants in the Bronx. Memphis participants called most often to make sure their pay- 
ments would be posted that month, and to find out the amount. MDRC verified that most 
participants received the money they were owed for the rewards they had completed, and that 
the payments were mostly accurate. There were a few cases of overpayment, but generally if 
participants did not get paid for rewards they had earned it was because they did not have 
valid bank accounts or stored- value cards. ^ 


fyhe payment-processing team accepted mail for 7 or 8 days after the 15th to capture as much of the 
postmarked mail as possible, especially mail from Memphis, which took longer to travel to New York City. In 
Year 1, CAS-Central noticed that its mail was not being delivered expeditiously and began going to the post 
office to pick it up. 

^In Year 1, the majority of participants who received payments had their earnings delivered to savings 
accounts (60 percent). Another 28 percent of participants had their earnings delivered to checking accounts, 
while only 2 percent of participants used stored-value cards. The remaining 10 percent of participants used a 
combination of savings, checking, and stored-value cards. These figures were nearly identical in Year 2: 59 
percent used savings accounts; 28 percent used checking accounts; 2 percent used stored-value cards, and the 
remaining 1 1 percent used some combination thereof 

Stored-value cards work like prepaid debit cards. Although they have associated fees, account holders can 
use them to receive direct deposits, make cash withdrawals, and for point-of-sale purchases. Family Rewards 
encouraged participants to use a bank or credit union account to receive program payments and to help them 
establish a connection to the mainstream banking system. To facilitate this process, the program invited banks 
that offered no-fee, “safe” savings accounts to attend orientation sessions and market directly to families. It is 
likely that many of the savings accounts participants used to receive payments were these safe accounts. 
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SIF Family Rewards 
Figure 2.2 

Year 2 Earnings Statement 


Name 
Address 
Address 
Address 
Family ID - 


Your Family Rewards 
Earnings Statement is Here! 


FAMILY 

REWARDS 


New York Helpline: 212-994-2748 


Memphis Helpline: 901-405-7970 


A new year of Family Rewards has begun! 
Have you started to EARN? 


Students... Bring your first report 
card to your Advisor 

Review your report card with an Advisor and add 
up the money you'll earn. Stop by as soon as your 

report card is ready- you can earn up to $600 for 
good grades this year! 


Parents... Schedule dental visits now! 
The deadline for first visits is February 28 

If you and your family have not had dental 
appointments since year two began this September, 
schedule now! To earn the first dental reward, visits 
must be on or before February 28. 


K) 


Figure 2,2 (continued) 

This page shows reward and bank account information we have received through November 15, 2012, If you submitted information after that 

date, it will show up on your next Earnings Statement. 


The amountyour family has earned in Year Two. Congratulations! 


ID 

First Name 

Activity Type 

Activity 

Amount of Money 

I-XXX1 

XXX 

Health 

Semi-Annual Dental Visit 

$100 





Total: $100 


How could your family earn more? Beloware some rewards waiting for you. 
Call us - many issues are easy to fix! 


ID 

First Name 

Activity 

Type 

Activity and Time Period 

Why Didn’t 1 Earn? 

(Call us! We’ll help you submit information for 
activities you can still earn!) 

Amount of 
Money 

l-xxxl 

XXX 

Education 

Attendance- January 

Education Activity Not Completed 

$40 






Total: $40 


SOURCE: CAS's Family Rewards program materials. 





Marketing the Incentives 

The most important behavioral changes that occur due to Family Rewards happen outside of the 
neighborhood partner organizations (NPOs), through unmediated interactions that participants 
have in their families, and in educational, health, and workplace settings. The Family Rewards 
staff must therefore succeed in explaining the incentives, presenting them in a motivating way, 
and continuing to market them over time so that participants do not forget about or lose interest 
in them. They must also find ways to help participants navigate the resources in their communi- 
ties that can assist them in earning the rewards. 

Most families kept track of the types of rewards and the schedule for submitting paper- 
work by referring to the coupon book, and through conversations with their Advisors or the 
Helpline staff The coupon books were first distributed during group orientation sessions 
conducted by the Advisors at the start of the program. In Year 2, the coupon books were picked 
up individually or during a group session held during the summer before the start of the new 
program year.'’ The initial group orientation sessions tried to achieve a mix of motivation and 
education, and they were largely successful. High school students were encouraged to attend 
with their parents and the presenters often engaged them directly. The information about the 
coupons and schedule for submitting rewards was presented clearly, but in the group setting the 
staff did not try to get into the complicated issue of which documents would be required to 
verify different kinds of employment. Staff members encouraged parents to call or come in for 
individual appointments to address verification requirements for work situations not covered by 
the generic instmctions (and even for those that were covered). 

Throughout the program, the Advisors and CAS-Central used various tactics to keep 
the rewards on the minds of the participants. They discussed rewards during regular family 
guidance sessions, included flyers in the Earnings Statements or mailed them separately, hosted 
workshops and group events to expose participants to service providers in a particular domain 
(Appendix Table B.l), and conducted “campaigns.” The campaigns were periods of focused 
attention to particular reward categories. For example, in March 2013 the NPOs launched a 
campaign to target families who had not submitted Year 2 health coupons. Porter-Leath held an 
event where a dental partner was on-site to complete cleanings and Step Ahead Foundation, an 
organization that provides free health screening and birth control for women, was present to talk 
to teens about health and wellness. Starting in Year 2, the NPOs also began campaigns to give 
more support to participants who had coupon rejections. 


^Some were delivered to participants’ homes by staff members, especially in Memphis. 
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Did these efforts succeed in educating participants about the incentives and the submis- 
sion mles? The data collected from focus groups with participants suggest that they did/ 
Parents and students generally had no trouble remembering the incentives when asked to list 
them several months after orientation. As expected, participants who earned rewards most often 
had the best understanding of the rewards and payment process. In most cases, the biggest 
challenge was remembering the specific dollar value associated with the reward, especially for 
rewards like the one for grades in which the dollar value for many students in the Bronx 
changed between Years 1 and 2, or for rewards that were paid out infrequently, like the one for 
taking the SAT or ACT. Interestingly, although some parents had difficulty in proving the hours 
they had worked (as discussed below), it was not because they stmggled to understand the 
workforce reward. They knew what needed to be submitted, but some parents had difficulty 
obtaining the needed documents or achieving the benchmark of working 120 hours per month. 
Thus, reducing the number of incentives seems to have helped participants understand the 
rewards better, suggesting that coupon rejections, which most families experienced at least once 
(see below), were not primarily a function of poor understanding. 


Patterns of Reward Receipt in the Three Domains and by City 

Overall, families who earned rewards earned an average of $2,160 from the program in Year 2 
(Table 2.1). One of the short-term goals of a conditional cash transfer program is to reduce 
immediate material hardship by transferring cash to families. For a family of three with an 
income at the 2014 federal poverty level, the cash transfer from Family Rewards represents an 
11 percent boost in income. The impact evaluation will investigate whether this amount is 
sufficient to reduce hardship. One of the factors that will affect this outcome is the large 
proportion of household reward earnings that was paid to high school students for education 
rewards.* Among families in which students earned some rewards, about 57 percent of house- 
hold earnings — an average of $1,146 for the year — went into high school students’ bank 
accounts or stored- value cards. Students typically opened joint accounts with their parents, so it 
is not clear whether this income was used by parents for household expenses, was used by 
students to cover individual expenses, was split between the two, or was saved. How reward 
earnings were spent, and the question whether this change in the balance of earnings between 
parents and students created family conflict, will be investigated in the next report. At present, 
there are anecdotal reports from both cities that parents in the most economically disadvantaged 

^The 24-month survey will provide a more comprehensive analysis of the issue of participants’ under- 
standing of the incentives and verification procedures, as well as their responses to rejections. 

^Baseline survey data on grade levels were used to supplement school records data to determine the sam- 
ple of high school students. Students who were in eighth grade in Year 1 according to the baseline survey were 
included as high school students in the Year 2 sample. Additionally, students in twelfth grade in the Bronx in 
Year 1 were excluded from the Year 2 sample. 
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SIF Family Rewards 
Table 2,1 

Summary of Rewards Earned by Families in Year 2 


Outcome 

Bronx 

Memphis 

Total 

Family earned at least 1 reward (%) 

94.8 

97.9 

96.3 

Education reward 

89.8 

96.4 

93.1 

Attendance 

82.7 

94.6 

88.6 

State core exam 

68.9 

56.6 

62.8 

SAT/ACT 

9.1 

3.3 

6.2 

Report card 

78.0 

82.4 

80.2 

Health reward 

79.1 

78.5 

78.8 

Annual physical 

72.3 

68.5 

70.4 

Biannual dental 

70.0 

71.5 

70.7 

Workforce reward 

45.5 

41.9 

43.7 

Full-time employment 

45.4 

41.3 

43.3 

GED certificate 

0.5 

1.1 

0.8 

Among families who earned at least 1 reward 




Average reward amount earned'^ ($) 

2,281 

2,043 

2,160 

Percentage of earnings from student rewards * 
Average reward amount earned, by domain ($) 

58.1 

56.1 

57.1 

Education 

1,287 

1,014 

1,146 

Health 

617 

706 

661 

Workforce 

1,140 

1,117 

1,129 

Family picked up coupon book (%) 

95.5 

94.5 

95.0 

Sample size 

617 

613 

1,230 


SOURCE: MDRC calculations using CAS's Family Rewards program data. 

NOTES: Sample sizes may vary because of missing values. 

Year 2 covers reward activities that occurred between September 2012 and August 
2013. 

Italics indicate outcomes calculated for a subset of the full sample. 

““The lowest and highest amounts earned in Year 2 were $38 and $7,250 for the Bronx 
sample, $40 and $6,680 for the Memphis sample, and $38 and $7,250 for the full sample. 

’’Percentage of earnings from student rewards is calculated using education rewards 
earned by high school students among families who earned rewards and had students who 
earned education rewards in Year 2. 
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families took control of their children’s earnings. Students in these families may have been less 
motivated by the program because they did not directly reap the financial benefits of their 
achievements. 

Families demonstrated strong engagement in the program. Ninety- five percent received 
their coupon books in Year 2 , most by picking them up at the NPO. Ninety-six percent earned 
at least one reward from the program. The pattern of earning in each of the domains indicates 
that families earned both coupon-based and non-coupon-based rewards. Ninety-three percent of 
families earned at least one education reward, 79 percent earned a health reward, and 44 percent 
earned at least one workforce reward. Most families earned an attendance (89 percent), report 
card (80 percent), and medical or dental checkup reward (79 percent). The rewards for students 
for taking the SAT/ACT, and for adults for earning a GED certificate were extremely underuti- 
lized, with only 6 percent and about 1 percent of families earning these rewards, respectively. 

The levels of reward receipt by city were not notably different, except in cases where 
participants’ ability to earn a reward was strongly influenced by external factors like access to 
service providers (as was the case, for example, for the reward for adult dental care in Mem- 
phis), or the different ways the same eligibility mles played out in the two cities (as was tme of 
the reward for attendance, for example). In other words, considering the differences between the 
two cities in family characteristics and local context (described in Chapter 1), Bronx and 
Memphis participants were remarkably consistent in the degree to which they participated in the 
program. They even described similar contributing factors and barriers to earning rewards, 
which are outlined below for each domain. 

Earning Education Rewards 

The study sample included 754 high school students who were eligible for education 
rewards in the Bronx in Year 2, and 869 in Memphis, a difference of 115 students. Table 2.2 
shows that a slightly higher percentage of Memphis students earned at least one education 
reward (95 percent compared with 87 percent in the Bronx). Memphis students were more 
likely to earn the rewards for attendance and grades than those in the Bronx, but less likely to 
earn the reward for state core exams, which is worth much more. As a result, students who 
earned rewards in Memphis took home $340 less on average than students in the Bronx. Still, it 
is important to note that students in the two cities earned almost exactly the same number of 
education rewards per person (about 21) with about the same frequency (for example, roughly 
the same percentage claimed at least one education reward in six or seven months of the 
program year). 
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SIF Family Rewards 
Table 2,2 


Education Rewards Earned by Students in Year 2 


Outcome 

Bronx 

Memphis 

Earned any education reward (%) 

87.3 

94.5 

Among students who earned at least 1 education reward 

Average number of rewards earned 

21.0 

21.9 

Average number of months student earned reward 

7.0 

7.4 

Average total amount earned^ ($) 

1,057 

717 

Student earned at least 1 education reward in (%) 

0 months 

12.7 

5.5 

1 month only 

6.6 

5.2 

2 or 3 months 

10.5 

9.0 

4 or 5 months 

13.7 

12.0 

6 or 7 months 

11.3 

14.7 

8 or more months 

45.2 

53.6 

State core exam passed (%) 

62.5 

44.5 

Number of exams passed (%) 

0 

37.5 

55.5 

1 

28.7 

31.1 

2 or more 

33.8 

13.5 

Attendance reward earned (%) 

78.1 

91.5 

Distribution of number of attendance rewards earned (%) 

0 

21.9 

8.5 

1 or 2 

16.6 

14.4 

3 or 4 

11.3 

13.5 

5 or 6 

13.4 

16.1 

7 or more 

36.9 

47.5 


(continued) 
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Table 2.2 (continued) 


Outcome 

Bronx 

Memphis 

Report card reward earned’’ (%) 

74.8 

80.0 

Distribution of number of report card rewards earned (%) 

0 

25.2 

20.0 

1-3 

5.7 

2.1 

4-6 

6.9 

4.3 

7-9 

6.0 

5.3 

10-12 

7.2 

6.8 

13-15 

5.7 

8.9 

16-18 

6.6 

10.8 

1 9 or more 

36.7 

41.9 

Student earned at least 1 reward, by letter grade (%) 

A 

61.1 

67.7 

B 

70.4 

76.3 

C 

63.3 

77.3 

Among students who earned at least 1 grade reward 
Number of rewards earned by letter grade 

A 

5.7 

4.9 

B 

7.4 

6.3 

C 

3.6 

6.8 

SAT/ACT (%) 

1.1 

2.3 


Sample size 754 869 


SOURCE: MDRC calculations using CAS's Family Rewards program data. 

NOTES: Sample sizes may vary because of missing values. 

Year 2 covers reward activities that occurred between September 2012 and August 
2013. 

Italics indicate outcomes calculated for a subset of the full sample. 

^The lowest and highest amounts earned in Year 2 were $10 and $2,640 for Bronx 
participants, and $10 and $1,900 for Memphis participants, respectively. 

’’Report card coupons were awarded for each individual qualifying grade. 


Implementing the Education Rewards 

Some of the difference in the rates at which students earned rewards is related to the 
two cities’ school district policies, which led the cities to have different ways of verifying 
activities and different opportunities to earn rewards. Ninety-two percent of Memphis students 
earned the reward for attendance compared with 78 percent of students in the Bronx. Almost 
half of the students in Memphis earned the attendance reward 7 or more times out of the 10- 
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month school calendar, compared with 37 percent of Bronx students. Even though students in 
both cities were expected to attend school 95 percent of the time to qualify for the reward, 
Memphis City Schools maintains attendance records in a way that made it easier for Memphis 
students to demonstrate excellent attendance. In Memphis, tardiness does not count against the 
number of days attended. In contrast, in the New York City Department of Education each 
school maintains a record of tardiness, and being late twice equals one day missed. At the end 
of Year 2, CAS-Central stopped counting tardiness against attendance in the Bronx because the 
Department of Education informed the organization that these records are unreliable. This mle 
change was implemented at the start of Year 3. 

Different opportunities to earn rewards affected student earnings for the state core ex- 
ams. In New York City, students are well informed about the Regents exams. They understand 
their importance and take practice exams in class. They also have the opportunity to take each 
exam more than once until they pass or to improve their score. In Memphis, Tennessee’s new 
End of Course exams were being instituted for the first time at the start of Family Rewards. 
Many students did not clearly understand the importance of these exams. They were delivered 
at the end of a course as a “final exam” that factored into students’ overall course grades, and 
students could not determine how well they had done until their raw scores were scaled by the 
state. In addition, the state did not introduce all of the anticipated End of Course exams on 
schedule. The Family Rewards program designers had expected students to be offered 10 
exams. By Year 2, the state had only implemented 7. In response, the program designers 
increased the dollar amount for each End of Course exam Memphis students passed from $200 
to $300 to create more parity between the cities in potential reward earnings. 

In Year 2, a larger proportion of the Memphis student sample earned the reward for 
grades (80 percent compared with 75 percent for the Bronx), and Memphis students earned 
rewards for grades more often. There was a major change in the policy for this reward in Year 
2. The program designers decided to create this reward to provide an incentive for educational 
“inputs” (behaviors that lead to academic milestones like passing standardized tests or graduat- 
ing), and to give Advisors a chance to monitor students and provide continuing support for their 
performance, especially for students who were less academically prepared. It was assumed that 
most students received four report cards a year, so in the first year of the program the coupon 
book contained four coupons for this activity, and a student had to have each coupon signed by 
an Advisor to receive payment. At each meeting, the Advisor was expected to make suggestions 
about how the student could improve his or her grades, and provide general encouragement. 
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The reality did not align with these expectations. In New York City, schools decide the 
number and timing of report cards, so students in the Bronx received up to eight report cards. 
Even though they could take the subway on their own to meetings, students saw it as a burden 
to come into the NPO to get their report cards signed so frequently. Most schools in Memphis 
issue report cards on a quarterly schedule, but attending any meeting was hard for families 
given the lack of public transit, and it was often parents who brought in report cards to be signed 
rather than students. It also became clear to the Family Rewards Collaborative and CAS-Central 
that the meetings between students and Advisors had little substance. Many Advisors simply 
signed the coupon and got families on their way, so the interaction was not achieving its 
intended goals. 

In response to this, the Family Rewards Collaborative removed the signature require- 
ment and instituted only one mandatory meeting in Year 2, called an “academic review.”'® 
Once students had completed the academic review, they could begin earning rewards for grades 
simply by filling out the coupons themselves and mailing in their report cards. The change in 
policy had both positive and negative consequences. It removed hassles associated with claim- 
ing this reward and may have enabled families to earn more frequently: 42 percent of students 
in Memphis and 37 percent of students in the Bronx claimed this reward 19 or more times 
(Table 2.2). 

On the other hand, it reduced the foot traffic into NPOs in Memphis, so Advisors could 
not monitor student performance as closely. Figure 2.3 shows the share of grade rewards that 
were paid for As, Bs, and Cs. A smaller percentage of the grade rewards paid in Memphis were 
for As than in the Bronx, but a much larger percentage were for Cs (38 percent compared with 
22 percent)." A single academic review may not have been adequate for the “C students” in 
Memphis to improve their grades. 

In addition, the policy change led to a high number of rejected grade coupons as fami- 
lies attempted to be paid for this activity before their academic review and received rejections 


®The dollar value for each grade was set based on the expectation that students would receive four report 
cards a year. When CAS-Central recognized that students were receiving different numbers of report cards, it 
prorated the amount of each letter grade so that the maximum amount each student could earn for this activity 
was the same in the two cities. This led to cases where students in the Bronx who received more than four 
report cards per year were paid between $10 and $16 for a B rather than the advertised $20, which may have 
been confusing. CAS-Central sent out letters to families to explain the prorated amounts. 

'®To eliminate the need to prorate grades, the group also standardized the amount of the reward for each 
letter grade and allowed families to submit as many report cards as they wanted until they hit a cap of $600. 

"Table 2.2 also makes this point. Of students in the Bronx who earned a grade reward, they claimed a 
reward for a C an average of 3.6 times, compared with 6.8 times in Memphis. It should be noted, however, that 
the two cities’ grading scales were different. In Memphis, the range of scores that constituted a C was larger 
than in New York City, while the range of scores that constituted a B or an A was smaller (see Table 1.1). 


37 



SIF Family Rewards 
Figure 2,3 

Report Card Grades Rewarded in Year 2, by City 

Grades rewarded in New York 



■ A DB SC 


SOURCE: MDRC calculations using CAS's Family Rewards program data. 

NOTE: Year 2 covers reward activities that occurred between September 2012 and August 2013. 
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for ineligible classes or grades. Table 2.3 provides summary data on the frequency of coupon 
rejections and their resolution. It shows that coupon rejections were a common phenomenon, 
experienced by 92 percent of Memphis families and 70 percent of Bronx families. After parents 
resubmitted some of these coupons and had some of the errors resolved by the payment- 
processing staff, a total of 90 percent of Memphis families and 62 percent of Bronx families still 
had at least one rejected coupon that was not resolved at the end of Year 2.'^ Coupons for 
grades and full-time work led to most rejections. Figure 2.4 shows that about half of Memphis 
families had at least one grade coupon rejected, and about as many had at least one that was not 
resolved. The strong pattern of earning despite these rejections suggests that the rejections did 
not discourage participation.'^ Memphis Advisors engaged in aggressive outreach (possibly 
triggered by the large number of rejected coupons at the start of the year), and did home visits to 
complete academic reviews. This illustrates one of the challenges program designers face: they 
aim to engage participants in productive interactions by making those interactions an additional 
condition for receiving the education rewards, but also wish to avoid making the program 
excessively burdensome on families or staff members. 


SIF Family Rewards 
Table 2,3 


Summary of Reward Rejections in Year 2 


Outcome (%) 

Bronx 

Memphis 

Families had at least 1 reward 



Submitted 

90.1 

97.6 

Rejected 

70.3 

92.2 

Resubmitted 

43.3 

44.1 

Rejected, resolved 

40.0 

42.1 

Rejected, unresolved 

61.9 

90.4 

Sample size 

617 

613 


SOURCE: MDRC calculations using CAS's Family Rewards program data. 

NOTES: Sample sizes may vary because of missing values. 

Year 2 covers reward activities that occurred between September 2012 
and August 2013. All reported submissions and rejections are only for 
coupon-based rewards. 


*^In this case “resolved” means that the error was addressed, the family or the staff member resubmitted 
the coupon, and the coupon was approved. 

*^The next report will use the 24-month survey to investigate this issue further. 
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SIF Family Rewards 
Figure 2,4 

Coupon Rejections in Year 2 

All coupon-based rewards 



Coupon outcome unresolved 


S Bronx ■ Memphis 


Report card rewards 



unresolved 

Coupon outcome 


S Bronx ■ Memphis 

(continued) 
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Figure 2,4 (continued) 
Full-time employment rewards 


100 1 



Submitted at least 1 At least 1 rejected 


Coupon outcome 

H Bronx ■ Memphis 



At least 1 rejected, 
unresolved 


SOURCE: MDRC calculations using CAS's Family Rewards program data. 

NOTES: Year 2 covers reward activities that occurred between September 2012 and August 2013. 
All reported submissions and rejections are only for coupon-based rewards. 


Families’ Perspectives on the Education Rewards 

The above discussion demonstrates that the pattern of reward earning is the result of both 
students’ investments in school-related behavior and the policies used to determine eligibility for 
the rewards. The education domain was also clearly a priority for most families in the program. 
In focus groups, parents typically described Family Rewards as “for their children.” Parents and 
students generally seemed comfortable with the concept of attaching financial incentives to 
academic achievement. A student in the Bronx described the ways her mother integrated an 
explicit discussion of the financial benefits of achievement into everyday conversation. 

Yeah, like 1 remember when I would wake up late my mom was like, “Hurry up 
and go so you don’t miss out [on rewards].” Or when 1 did really bad on a grade, 
well not really bad, 1 just got like a low B, she’d be like, “Oh, you still did good, 
but you know if you do better you’ll get this amount of money.” 
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It is not surprising, then, that when students and their parents attributed improved aca- 
demic performance to Family Rewards, it was usually because the student wanted to earn 
money from the program. The incentive made it more unpleasant for students to perform badly, 
and amplified the positive feelings associated with doing well. A student in Memphis said: 

It’s a motivator. It motivated me to — shoot, if I had a B, usually I’d just be like, 

“1 don’t care. I’ll just stay with the Bs, a B is good.” But now if 1 got a B, 1 want 
an A because 1 wanna make $30 instead of $20. 

One parent used the history of earnings to generate productive competition among her 
children. 


Well, my 17-year-old, he was making bad grades until he started seeing my other 
kids get their money. 1 be like, there’s some Ds, you don’t get paid. Look at their 
report card and look at yours. This is what they’re going to get every time. Guess 
what? Now he get it. 1 mean, it even brought his grades up. 

Another parent in the Bronx shared the following description of her son’s turnaround: 

When he saw that there was money that he could make ... he didn’t pass no class 
one semester, passed every class the next semester. His Regents, he passed two. 
Before, he passed none. So, every semester, every — when he gets that — with 
that money in his account, that’s the happiest. So 1 show [him], you know, I'm 
trying to show him. 1 said, “They’re encouraging you. You also have to encour- 
age yourself” 

As demonstrated by this parent, getting the financial incentive could be linked to non- 
monetary reasons to want to succeed in school. 

Aside from increased motivation, there was not much agreement among students and 
parents about other factors that contributed to better academic performance. They did, however, 
agree about the barriers to improved performance. These included students not getting along 
with teachers, lacking practical strategies to get to school on time or study for tests, attending 
disorganized schools with poor guidance counseling, and feeling unprepared for state exams 
based on the insfruction they had received during the semester. For students facing these 
challenges, the financial incentives did not carry much motivational force. Having tried to earn 
the rewards and failed, they stopped thinking about them. 

The money’s nice. You know, that’s nice, but it’s like, after a while, you know, 
say you try your hardest, but you still get low grades. After a while, there’s just 
no point. 

The program model includes guidance to help students develop strategies to address 
barriers to earning rewards, and develop resilience in the face of failure. The next chapter 
reviews how the Advisors approached these tasks. 
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Earning Health Rewards 

Families in Memphis tended to be larger because they had more children, creating an 
opportunity for these families to earn more in the health domain than families in the Bronx if all 
enrolled family members attended medical and dental checkups regularly. Table 2.1 showed 
that 79 percent of the families in the sample earned at least one health reward, but that number 
masks large disparities in adult earning between the cities. Only 49 percent of Memphis adults 
earned any health reward, compared with 67 percent of Bronx adults (Table 2.4). Three-quarters 
of adults in Memphis did not earn any rewards for dental visits. The figures among children are 
more comparable. Seventy-six percent of children in Memphis earned any health reward 
compared with 7 1 percent of Bronx children. More children in Memphis earned the dental 
checkup reward (71 percent compared with 59 percent in the Bronx), and more Memphis 
children went to the dentist twice (38 percent compared with 25 percent in the Bronx). In the 
end, Memphis families earned an average of $706 from the health rewards in Year 2 compared 
with $617 in the Bronx. 

From one perspective, health rewards were the easiest to earn. Advisors marketed these 
incentives as a “quick win” for families that could put hundreds of dollars into their accounts. A 
Memphis Advisor explained: “Taking the kids to the doctor wasn’t such a big challenge, so 
that’s quick money that they just got right off the bat.” To the extent that families were already 
in the habit of attending these appointments, it should have been relatively easy to get paid for 
the activity. Parents reported that 94 percent of children in the Bronx and 88 percent of children 
in Memphis attended a medical checkup the year before entering the program. There were 
similarly high reported levels of dental care for children. For adults, the proportion that reported 
having a medical checkup in the year before starting the program was somewhat lower but still 
high at 87 percent in New York, and 77 percent in Memphis. However, only 36 percent of 
adults in Memphis had attended the dentist during this period, compared with 78 percent of 
adults in the Bronx. 

From another perspective, earning health rewards could be considered very difficult be- 
cause these coupon-based rewards put organizational demands on the family. Parents had to 
have health insurance coverage or access to a low-cost clinic, make appointments for them- 
selves and their children on a particular schedule, remember to bring all of their coupons, get 
them filled out completely on the spot or arrange to pick them up at a later date, and mail them 
to Family Rewards. One academic commentary described this process as excessively burden- 
some for families facing the multiple sources of stress that come with poverty. 


'‘'Mullainathan and Shafir (2013). 
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SIF Family Rewards 
Table 2,4 

Health Rewards Earned by Participants in Year 2 


Outcome 

Bronx 

Memphis 

Parents 



Earned any health reward (%) 

66.7 

49.3 

Among parents who earned at least 1 health reward 



Average number of rewards earned 

2.0 

1.5 

Average total amount earned ($) 

199 

145 

Health rewards by category (%) 



Annual physical 

50.8 

41.1 

Dental care checkup 



0 dental checkups 

43.7 

75.9 

1 dental checkup 

30.5 

17.6 

2 dental checkups 

25.8 

6.5 

Sample size 

655 

631 

Children 



Earned any health reward (%) 

71.3 

75.9 

Among children who earned at least 1 health reward 



Average number of health rewards earned 

2.0 

2.2 

Average total amount earned ($) 

203 

218 

Health rewards by category (%) 



Annual physical 

60.8 

56.9 

Dental care checkup 



0 dental checkups 

41.2 

29.2 

1 dental checkup 

34.0 

33.0 

2 dental checkups 

24.8 

37.9 

Sample size 

1,481 

1,778 


SOURCE: MDRC calculations using CAS's Family Rewards program data. 

NOTES: Sample sizes may vary because of missing values. 

Year 2 covers reward activities that occurred between September 2012 and August 2013. 
Italics indicate outcomes calculated for a subset of the full sample. 
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Families’ Perspectives on the Health Rewards 

Participation in this area was strong for children, but participants earned the medical 
and dental checkup coupon at rates lower than the rates that they reported on the baseline survey 
they were already attending these appointments. This suggests that there were some barriers to 
earning.'^ Coupon rejections do not explain the discrepancy. A review of rejected coupons 
shows that coupon rejections in the health domain were far less frequent and more likely to be 
resolved than rejections for grades and full-time work (not shown in a table). The key barriers 
described by participants and program staff members were lack of dental coverage for adults, 
lack of personal organization, and fear. 

A higher proportion of the Memphis adult sample started the program with no dental 
coverage than in the Bronx, and the public health insurance program in Tennessee does not 
cover cleanings, as it does in New York State. As a result, many adults in Memphis cited the 
out-of-pocket cost of dental care as the reason they did not earn the reward. The program staff 
took on this issue in Year 2 of the program through partnerships with providers. Staff members 
identified and established relationships with dental providers who offered cleanings for free or 
at a special price ($10 to $20). These providers included the Concorde Career Dental College, 
Christ Church Community Health Center, the Memphis Health Center, and the Firestone Dental 
Group, which has a mobile clinic. Memphis Advisors invited Firestone to provide cleanings at 
health fairs hosted by the NPOs, which were scheduled before dental coupons were due.''’ The 
Family Resource Fund was also used to cover some costs associated with getting dental care. 

This is an example of the program working on both the demand and supply sides of the 
preventive dental care issue. Conditional cash transfer programs have traditionally been thought 
of as demand-side interventions: they seek to improve outcomes by encouraging participants to 
take advantage of existing resources rather than creating new services. But in this case the 
NPOs had to go further than simply creating demand and actively remove barriers to access to 
the existing supply of dental providers. Otherwise the incentive would simply not have been 
used. As it was, the level of receipt in Memphis in Year 2 was still low compared with the 
Bronx, but slightly higher than it was in Year 1. 

The program also implemented several strategies to try to help participants deal with the 
organizational challenges associated with claiming the health coupons, and with their anxiety 
about going to see a doctor or dentist. CAS-Central and Advisors reached out to participants by 
phone and letter, and through automated calls, to remind them about upcoming deadlines for 

'^Baseline reports may also have been inflated by social desirability (the tendency to answer questions in a 
way that comports with social norms) and recall issues. 

'^The health fairs also provided additional information. NPOs invited community partners to host booths 
about public health insurance, food assistance, and foster care, among other issues. 
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health coupons. In conversations with participants, staff members would acknowledge their 
fears and help them weigh the costs and benefits of taking control of their health status. One 
adult participant described how this helped her earn the medical checkup reward after seven 
years of avoiding the doctor. 

[Going to the doctor] was kind of scary. 1 wasn’t really going to the doctor like 
that, so you all kind of helped me got there. 1 was scared to find out if 1 had dia- 
betes or high blood pressure. Really, 1 was scared.... But 1 actually did it ... 1 
have no problems, and 1 been waiting all these years. Y ou know what I’m say- 
ing? And it really wasn’t nothing to it. So it was the motivation that helped me 
go, really. 

Note that if the problem of fear of medical professionals was as pervasive as partici- 
pants and staff members have suggested in qualitative interviews, the self-reported baseline data 
about participants’ frequency of doctor and dentist visits are probably overestimated. 

Earning Work Rewards 

Compared with Advisors in Memphis, Advisors at the Bronx NPOs each had a slightly 
larger caseload of adult participants on average, because of the higher number of married 
couples enrolled (that is, families with two enrolled adults in the program rather than one). As 
reported in Chapter 1, about half of the Bronx adults are immigrants, speak Spanish as their 
primary language, and did not have a high school diploma or GED credential at the start of the 
study. Despite these potential barriers to employment, 57 percent of these adults were working 
when they joined the study, compared with 44 percent of Memphis adults. Almost all house- 
holds in the study were receiving food stamps when they enrolled in the study. Only 5 percent 
of families in the Bronx also received cash assistance, compared with 3 1 percent in Memphis. 
The unemployment rate was similar in Memphis and New York City in 2013, but median 
wages were lower in Memphis than in the New York area. 

These differences between cities did not lead to any meaningful differences in the 
proportion of adults who earned work rewards, the frequency with which they earned them, or 
the amount they earned. Forty- four percent of Bronx adults and 40 percent of Memphis 
adults earned the full-time work reward in Year 2 (Table 2.5). Most earned it consistently: 
among those who earned at least one, the average number of rewards earned was seven. 
Faming the reward correlated with working status at the start of the study. About 77 percent 
of adults who were working full time when they enrolled in the program earned the reward. 


'This section focuses on the reward for full-time work because very few participants earned the GED 


reward. 
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SIF Family Rewards 
Table 2,5 


Workforce Rewards Earned by Parents in Year 2 


Outcome 

Bronx 

Memphis 

Earned any workforce reward (%) 

43.7 

41.1 

Among participants who earned at least 1 workforce reward 

Average number of rewards earned 

7.4 

7.3 

Average total amount earned ($) 

1,120 

1,108 

Work rewards by category 

GED certificate (%) 

0.5 

1.1 

Full-time employment (%) 

43.5 

40.4 

Parent earned at least 1 full-time work reward in (%) 

0 months 

56.5 

59.6 

1 to 3 months 

7.2 

7.6 

4 to 6 months 

11.0 

9.0 

7 or more months 

25.3 

23.8 

Among parents who earned at least 1 full-time work reward 

Average number of months parent earned reward 7. 5 

Earned a full-time work reward, by employment status at random assignmenf* (%) 

7.4 

Employed full time at random assignment 

73.7 

79.5 

Employed part time at random assignment 

37.9 

45.3 

Not employed at random assignment 

18.7 

16.7 

Sample size 

655 

631 


SOURCE: MDRC calculations using CAS's Family Rewards program data and baseline 
survey data. 

NOTES: Sample sizes may vary because of missing values. 

Year 2 covers reward activities that occurred between September 2012 and August 2013. 
Italics indicate outcomes calculated for a subset of the full sample. 

“Baseline characteristics are provided only for parents who reported their employment 
status on the baseline survey. 


compared with only about 42 percent of adults who were working part time, and about 18 
percent of those who were unemployed. Almost no adults earned the GED reward, even though 
48 percent of adults in the Bronx and 31 percent in Memphis did not have a high school 
credential at the start of the study. 
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Implementing the Work Reward 

Although the criterion for earning the work reward was straightforward — working 120 
hours per month — participants often had difficulty with the verification procedures. The third 
panel of Figure 2.4 shows that most people who submitted a full-time work coupon had at least 
one rejected, and about three-quarters of that group had at least one coupon that was never 
resolved. Does this mean that hassles associated with verifying work stood in the way of some 
participants understanding this reward, or receiving the payments they were entitled to? 

These barriers may have existed for some participants, although it is impossible to esti- 
mate the size of this group without a larger survey. In addition to the regular focus groups with 
parents, MDRC investigated the role of verification hassles in earning the work reward by 
interviewing parents who had had a coupon for full-time work rejected. The reasons for having 
a coupon rejected were, in order of frequency: not having enough hours, missing pay stubs, pay 
stubs that did not align with the program schedule, and missing or incorrect documentation. 

Some participants submitted coupons even when they did not have enough hours be- 
cause they felt they could not get more hours from their employers, were close to the threshold, 
and hoped to be paid.'"^ Some experienced rejections regularly that were caused by a mismatch 
between their jobs’ pay periods and the program schedule. This initially caused a great deal of 
confusion for participants. A participant in Memphis addressed the problem by sending in all of 
her pay stubs every month. She said. 


** Self-employed workers were also eligible for this reward. The program required that participants submit 
proof of hours worked or a representation of hours worked along with proof of quarterly or annual taxes filed. 
Midway into Year 1, the program established an earnings threshold as a proxy for full-time employment for 
those participants whose only record of income was the money reported on a tax return. That threshold is the 
federal minimum wage ($7. 25/hour) times the 120 hours required by the program, which amounts to $870 per 
month. 

'’in focus groups, many parents suggested that the full-time work coupon should have tiers below 120 
hours in order to provide an incentive for parents who worked part time by choice or could not get more hours 
at their current jobs because of their employers’ decisions. For example, home attendants or child care 
providers might have varying schedules depending on the needs of their clients. One participant in New York 
who worked as a home attendant explained: “When your client goes in the hospital, and you’ve got to wait for 
[additional] hours ... your hours start to fall, and you have not 120 hours, and that really gets on my nerves.” 
Some participants worked part of the year or had their hours reduced for a variety of reasons. For example, 
people working according to the school calendar do not work during school vacation time. One participant in 
Memphis who was a bus driver explained: “I don’t always get 30 hours a week. Or you know, make the 
equivalent of — but I work every week. It may be 27, 28 hours, and I can’t get [30 hours].... Yeah, that’s the 
only thing I don’t like it. I think the hours should be cut a little bit.” These participants expressed fhistration 
about the program’s hour threshold rather than seeing it as an incentive to take on additional work or seek out a 
job that would provide full-time hours. 


48 



One thing hard is the work, because 1 think we get paid twice a month. That’s 
hard. Because you’ve got to send in three [pay stubs]. So that’s something I’m 
having a problem with. Because 1 send in — 1 don’t know what 1 be sending in, 1 
just be sending in all of them. 

The Helpline staff and Advisors ended up recommending that participants adopt this 
strategy across the board: sending in all pay stubs and letting the payment-processing staff sort 
it out. The result of this process was that many people experienced rejections regularly that were 
resolved the next month when they sent in pay stubs that brought their hours for the prior month 
over the payment threshold. Because the payment-processing staff completed the submissions 
on their own without requiring the participants to resubmit their coupons, participants often had 
more rejections than they actually knew about. 

Another paperwork-related issue that some participants encountered was receiving pay 
stubs that did not show the hours they had worked. (This happens with salaried employees, for 
example.) This situation required participants to take an additional step to earn rewards: they 
had to go to the human resources department to request documentation of the hours they had 
worked for the month. One participant had to go to her human resources office every month. 
She said: 


Sometimes if 1 go to the office [human resources] to ask, it’s taking like, they’re 
gonna make me crazy. “Why you need this? Why you need it? Why you need 
it?” And they keep on asking, so.... Ugh. 1 have to bring it to office to ask them ... 

1 go there and tell them 1 need it. They give it to me. 

This could be a particular hassle since the human resources department was not always 
near where the person worked. Family Rewards made an adjustment to the paperwork require- 
ment to address this situation. For those receiving a base salary, the program asked for a one- 
time submission of documents verifying the number of hours they worked. This provision, 
however, did not account for those who got paid by the hour but whose pay stubs did not 
include hours worked.^” 

Paperwork for claiming workforce rewards was particularly challenging for the self- 
employed. Self-employed participants had to wait until they filed their annual tax returns or 
submitted quarterly estimated tax returns in order to claim workforce rewards. They also had to 
provide proof of the hours they had worked or show earnings over a wage threshold. To 
facilitate this process, CAS-Central created calendars and logs to help self-employed partici- 
pants track their hours on a weekly basis. 


^**Over time, CAS-Central extended the wage-threshold-verification option created for self-employed 
workers to traditional workers who faced an undue burden getting documentation from their employers. 
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Personal organization was also a barrier for participants in trying to keep track of their 
pay stubs given the multiple and unpredictable challenges they faced. One participant responsi- 
ble for the care of her mother who has Alzheimer’s noted: 

Fm taking care of my mom. She just came down with Al z heimer’s. So it’s hard 
to, like, keep my pay stubs together in one envelope. 1 try to do it so much, but 
then when 1 come and 1 think 1 have, you know, all the dates of the pay stub, 

[staff members say], “Well, you’re missing ... days.” Oh, that’s so finstrating 
when 1 come and think that 1 have them all ... cause like sometimes you get paid 
every week, you put a pay stub here, you put a pay stub there. So my main goal 
now is to try to have one big envelope and just put them all ... every week in the 
same envelope so that 1 can just keep track of them. 

Advisors provided guidance on how to get organized and keep track of paperwork. For 
example, one Advisor coached a family on how to use a binder to organize paperwork for each 
member of the family. Advisors also launched outreach campaigns over several months to 
provide support to parents who had workforce coupons rejected. 

When parents were asked during interviews and focus groups how they felt about re- 
jected coupons, the majority of participants felt that the decision had been fair because it had 
been in line with the program mles. While disappointing, they accepted coupon rejections 
because they were based on the mles of the program and they had to comply with the mles. 
Many simply resubmitted the coupon hoping it would be approved the second time. The 
implementation research found, as well, that CAS-Central monitored the issue and took steps 
whenever possible to clarify or alleviate unnecessary documentation requirements while still 
ensuring that the reward was only paid to those who met the hours threshold. This was a 
challenging task, which CAS-Central performed successfully. 

Families’ Perspectives on the Work Reward 

The bigger question, then, is why more participants did not respond to the full-time 
work incentive. The implementation research suggests four main reasons. First, some partici- 
pants cited disabilities that did not permit them to work full time.^' Second, some participants 
simply thought that it would be very difficult to find a job in the current economic climate. 
When asked whether the workforce reward provided a motivation to seek full-time employ- 
ment, a participant said: “Oh no, you’ve got to stick with what you’re with. You can’t go get 
another [job] because it’s impossible right now.” The sense of the impossibility of improving 
their work situations or fear of the consequences of change kept many participants in place. 


^’sixteen percent of adults in the Bronx and 18 percent of adults in Memphis reported having a physical 
disability that limited their ability to work, while 7 percent of adults in the Bronx and 8 percent of adults in 
Memphis reported that a psychological issue limited their ability to work. 
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Third, some participants had child care or other family responsibilities like caring for elderly 
parents that they made a higher priority than seeking full-time employment. One participant in 
Memphis, for example, noted: 

1 think 1 probably turned in probably one work paper, because by me driving the 
school bus, when my kids are out. I’m out, and 1 don’t have to worry about no- 
body watching them, or if they get to the house when I’m at work, or wondering 
who’s in the house with them, who they let in. So the way my schedule is with 
my kids. So me trying to go somewhere else would be totally off, because by the 
time 1 get off work, 1 can take my kids to school. So trying to find something else 
right now is not [a possibility]. 

Finally, some participants were concerned about the potential effect of full-time em- 
ployment on their benefits. When a participant in the Bronx who was working part time was 
asked why she had not earned the work reward, she responded: 

No, in reality. I’m the one that doesn’t want to make that amount of hours. So, 1 
have a comfortable schedule that does not meet the required hours. So, no, 1 have 
not earned that money and 1 don’t think I’ll get it.... So, I don’t think I’ll get it 
because 1 have other form s of assistance that 1 don’t want to lose in order to earn 
[work rewards].... 1 have done my calculations to see what’s best and 1 think this 
is the best. 

In the second year of the program the NPOs, with guidance from CAS-Central, made a 
concerted effort to create partnerships with organizations offering job-readiness programs so 
that parents who were willing to consider full-time employment would have the resources 
needed to make the transition. In the Bronx, the staff developed a relationship with Workforce 1 
Career Centers. Career Centers identify job openings and advertise them to suitable job seekers. 
While job seekers with barriers to employment can register at the centers, the NPOs were 
advised that the centers focus on meeting employer needs by identifying candidates who have a 
minimum level of work experience and other signs of job readiness. The Bronx staff participat- 
ed in training to become official community partners to the Career Centers. Staff members 
learned how to conduct an in-depth assessment of a participant’s barriers to employment and 
how to determine if she was eligible for available jobs. If she was, a staff member would refer 
her to a local Career Center with a referral ticket, which entitled her to attend recmitment 
events. If an individual was not job-ready, a staff member could refer him to other community- 
based organizations that would work on addressing his barriers to employment. Bronx Advisors 
did not use Career Centers very much, because based on the responses of a few parents they 
referred to them, they came to believe that individuals with low levels of education or poor job 
histories would not be well served there. Nevertheless, the staff found it beneficial to learn more 
about the services available for non-job-ready individuals. Staff members in the Bronx also built 
relationships with other workforce-related organizations around the city like LIFT and Henry 
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Street Settlement, and with training programs like the Cooperative Home Care Associates, 
which provides free training for home health aides. 

In Memphis it was more challenging for NPOs to find partner organizations because 
most workforce programs there were reserved for cash-assistance clients. Porter-Leath and 
Memphis HOPE invited JobLinc, a program of the public library, to on-site events to recruit for 
its mobile job-readiness service, but these events were not well attended. The Memphis Advi- 
sors also took it upon themselves to send out regular job lists, and learned to assist participants 
with some job-readiness skills, like writing resumes. 

Chapter 3 describes how the program confronted the pervasive problem of resistance to 
(or ambivalence about) working full time through particular guidance techniques. This occurred 
because CAS-Central’s technical assistance staff recognized that referral partnerships and job- 
related services could only be effective if parents were willing to take advantage of them. 


Conclusion 

Memphis and Bronx participants showed a fairly consistent pattern of reward receipt, a positive 
indication that it is possible to replicate the original conditional cash transfer program. Memphis 
participants were not at a disadvantage overall in earning rewards. The biggest discrepancy 
between the cities was in the number of adults who earned the reward for preventive dental 
care, and to address that discrepancy the Memphis NPOs began to facilitate access to low-cost 
providers in Year 2. On the other hand, it is a concern that approximately the same, low propor- 
tion of adults in the two cities earned the reward for full-time work despite differences in the 
demographic characteristics of the study samples from those two cities, and in the service 
environment and economic conditions surrounding them. 

The program operators acted quickly to address participants’ barriers to earning re- 
wards, at times modifying eligibility criteria or engaging in new campaigns to help families 
earn. For example, the program attempted to ease access to the reward for report card grades by 
removing the Advisor signature requirement at the start of Year 2. Open questions remain, 
however, about the best way to design the grade reward so as to balance the need for meetings 
to monitor performance with the need to eliminate barriers to claiming the reward for economi- 
cally distressed families. 
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Chapter 3 

The Family Guidance Component 


Introduction 

The biggest innovation in the design of Family Rewards 2.0 was the addition of the family 
guidance component. By incorporating an active, personalized support component, this version 
of Family Rewards breaks with a core concern of the first program. Family Rewards 1.0 was 
designed to test the power of financial incentives alone. It had staff members at each neighbor- 
hood partner organization (NPO) whose mandate was to provide “customer service” rather than 
“case management.” Staff members would generally wait for participants to initiate contact and 
focus the interaction on issues directly related to earning rewards rather than reaching out to 
participants and walking them through strategies for behavioral change. 

The results of Family Rewards 1.0 revealed that two types of participants in particular 
might benefit from additional support to help them earn particular rewards: (1) students who 
started high school scoring below grade level on eighth-grade math and English exams (“non- 
proficient” students), and (2) adults who did not have a high school diploma or General Educa- 
tional Development (GED) certificate when they entered the study. The impact findings from 
that earlier study found that although the program had substantial positive effects on the school 
outcomes of students deemed proficient when entering high school, it had no effects on non- 
proficient students. The study also found that the program may have somewhat worsened the 
labor market outcomes of parents who were more disadvantaged at the start of the program, 
particularly those without a high school credential. Thus, in Family Rewards 2.0, these two 
groups were identified as likely to need special outreach and support through the guidance 
component in order for the program to be more effective than the earlier model. 

More generally, the goal of family guidance in Family Rewards 2.0 is to extend the im- 
pact of the incentives to a broader segment of the program group, and to produce longer-term, 
more durable effects on behavior and financial well-being. The program designers believed that 
families would benefit from a regular source of support within the program, but aimed to 
structure that support in a way that aligned with the spirit and concept of the conditional cash 
transfer, and that was affordable given the constraints of the program budget. They did not 
require social workers or incorporate a full case-management system. Instead, the designers 
created a “triage” model that could be implemented by a team of paraprofessionals in which all 
participants would receive some guidance, but those who were having the most trouble earning 
rewards would receive more intensive outreach and support. A key question is thus whether the 
triage model was implemented as designed and how well it functioned to connect participants 
(adults and students) to the resources and effective advice they needed to earn rewards. 
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Key Findings 

• While the goals of family guidance stayed consistent, the strategy evolved 
considerably between Year 1 and Year 2. In Year 1, the guidance interaction 
was focused on relationship building and customer service. These were im- 
portant achievements, but they did not align completely with the original 
goals of guidance since the Advisors did not target participants who were 
earning few rewards (hereafter called “low-earning” participants), nor were 
they having deep conversations about barriers to earning rewards. 

• In Year 2, staff members were trained in motivational interviewing, a client- 
driven approach that changed the texture of family guidance sessions. By the 
second half of the year, nearly all staff members, regardless of credential lev- 
el, had achieved basic competence in this technique. This required a signifi- 
cant investment of professional development resources by the Children’s Aid 
Society (CAS)-Central. 

• The Advisors targeted low-earning participants with more guidance sessions 
in the second half of Year 2, as the triage model envisioned. The Advisors 
expended a tremendous amount of effort reaching out to these participants. 

• Advisors did not make many referrals to outside resources during the guid- 
ance sessions either before or after the introduction of motivational inter- 
viewing. They also did not make much use of the Family Resource Fund. 

• Advisors in the Bronx and Memphis implemented the family guidance com- 
ponent in different ways. In Memphis, the main barrier to participation in 
guidance was transportation. Staff members adapted by conducting guidance 
sessions over the phone or making home visits. They did not have much di- 
rect contact with students, and participants in Memphis tended to feel that 
they were not as close to their Advisors as did participants in the Bronx. In 
the Bronx, the Advisors saw students and adults more regularly, and devel- 
oped closer relationships with them. 

This chapter is organized into three sections. The first examines the core elements of the 
original model of family guidance. The second describes staff qualifications, training, and 
organizational support for one-on-one engagement. The third section compares the style of 
family guidance in Year 1 with that in Year 2 and includes an analysis of data from the pro- 
gram’s management information system. 
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Core Elements of the Triage Model 

The original, triage model of family guidance in Family Rewards 2.0 was designed to provide 
guidance proportionate to the participant’s need. In Family Rewards 1.0, the implementation 
study found that some families were entirely capable of earning the rewards on their own with 
only customer-service assistance. Other families, though, did not earn rewards and were not 
engaged in the intervention. These families tended to be the most disadvantaged when they 
entered the study. In the middle were families whose circumstances and ability to earn rewards 
varied. For these participants, some activities were within their grasp; their challenge was to 
earn rewards consistently and to take the right steps to earn rewards that required more effort or 
planning to obtain. The triage approach introduced a way to help more participants earn rewards 
and achieve important benchmarks more consistently, within the constraints of a modest 
program budget for the guidance function. 

As noted earlier, the new family guidance component was added to expand the propor- 
tion of participants earning rewards and to maintain strong levels of reward receipt. The 
intention was for Advisors to build tmst with participants, create a plan for earning rewards with 
each family at the start of the program, use earnings data to assess families’ progress toward 
maximal earning, and make referrals for those who needed them. This section considers the 
core elements of the original family guidance model in detail. 

Advisor Qualifications 

Advisors were expected to have general skills and to serve large caseloads. The NPOs 
were not expected to hire trained social workers or job developers to work as Advisors to 
families. Each NPO was required to hire three Advisors and designate each as either a “senior” 
or “junior” Advisor. The senior Advisors were expected to have somewhat more experience and 
to be able to offer support to the junior Advisors as they developed their skills. Each Advisor 
was to serve a stable caseload of approximately 100 families. This ratio was seen as adequate to 
allow Advisors to meet with the adults and high school students in every family twice per year. 

The Family Earning Plan 

Enrolled parents and high school students had formal guidance sessions with Advisors 
twice a year, organized around a dynamic document called a Family Earning Plan. The Family 
Earning Plan was expected to play a significant role in shaping the guidance conversation. The 
Family Rewards Collaborative sketched out the contours of the document and Seedco filled in 
the details. Appendix Figure C.l displays the first page for adults and students. Appendix 
Figure C.2 shows a summary worksheet that adds up the potential earnings for the whole family 
in one place. 
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The Family Earning Plan was meant to serve several purposes. First, it was an educa- 
tional and motivational tool that reinforced the messages the staff delivered during group 
orientations. In Family Rewards 1.0, participants stmggled to recall the incentives they could 
earn less frequently and were sometimes unclear about the conditions they had to meet to earn 
them. The Family Earning Plan provided an additional opportunity for Advisors to review the 
rewards and conditions, and to personalize the information by using family size and composi- 
tion to calculate the number of opportunities to earn. The document calls for the Advisor to sum 
up the maximum amount participants could earn. It was hoped that the large potential earning 
figures would impress participants and harness their drive to actually earn the rewards. 

Second, the Family Earning Plan was a self-assessment tool. It asked participants when 
they expected to be ready to earn, on three different time scales. Participants who chose the time 
scale furthest in the future were indicating that they had barriers to earning rewards or were not 
fully committed to doing the activity. This should have triggered a conversation with the 
Advisor about ways to become ready to earn. Third, the Family Earning Plan was a planning 
tool. Advisors were expected to make referrals in response to the discussion of barriers. The 
names of the referral agencies and a timetable for taking advantage of the services they offered 
would be written into the Family Earning Plan to build a sense of accountability. 

Advisors were asked to create Family Earning Plans during their initial meetings with 
families, shortly after orientation. Once filled out, one copy of the Family Earning Plan would 
be given to the family and another stored at the NPO. The model then called for the Family 
Earning Plan to be reviewed biannually with all enrolled parents and high school students. The 
Family Earning Plan gave students their own documents and regular check-ins, because Family 
Rewards 1.0 revealed that high school students need direct and simple information about the 
rewards they can receive and frequent reminders. 

Active Outreach 

Participants who were not earning consistently were identified using reward payment 
data and targeted for outreach. The model called for the caseload to be stratified based on 
earning history. Participants who fell below a certain earnings level would be targeted for 
outreach, which would consist in most cases of a Family Earning Plan review. It was challeng- 
ing from the start to develop outreach criteria that targeted the right participants at the right 
intervals. Should NPOs conduct their outreach based on individual or family earnings? Should 
they take into account all rewards, or a select few? The Family Rewards Collaborative did not 
make specific proposals. The CAS-Central staff took the lead in doing this. In Year 2, CAS- 
Central proposed that Advisors reach out to participants who met the following criteria at the 
end of Febmary 2013: 

• Students with report card earnings below $75. 
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• Adults who had earned rewards in Year 1 but not yet earned any in Year 2. 

(The vast majority of participants earned rewards in Year 1. Those who did 
not were very disengaged, and this definition focused attention on stmggling 
participants who seemed most reachable.) 

Referrals 

Advisors were not expected to provide direct services but were instead intended to be 
“resource experts” who could build rapport with participants, identify their barriers to earning, 
and connect them to high-quality, local service providers. This is a core tenet of the conditional 
cash transfer model: it is a “demand-side” intervention that seeks to shape participants into 
better consumers of existing resources, not create new resources. Each NPO was instmcted to 
develop an extensive list of referral partners with support from Seedco and later from CAS- 
Central. 


The Family Resource Fund 

A small discretionary fund, called the Family Resource Fund, was available to assist 
participants in achieving the conditions to qualify for rewards. The Family Resource Fund had a 
similar function to supportive-services funding in the workforce system. Each family was 
allocated about $300, which could be used to pay for services that would help them earn one of 
the rewards. This fund was created to address the problem that all payments for activities are 
delivered after they are completed, but participants might need to invest money up front to cam 
the incentives: for example, to get private tutoring to qualify for the state exam reward, or to pay 
for a licensing exam to begin searching for work in a particular field. The availability of this 
fund was not advertised. Rather, it was a resource staff members could draw on if needs 
emerged in conversation with participants. 

Events and Marketing 

While one-on-one interactions were central to the engagement strategy, NPOs were ex- 
pected to supplement family guidance with group events and mass marketing. The Family 
Rewards Collaborative envisioned that the NPOs would continue to engage the participants 
through multiple forms of outreach. In light of the large number of participants in each Advi- 
sor’s caseload, group events and mass mailings were seen as a way to stay in touch with 
participants between biannual meetings, provide useful information, and keep Family Rewards 
on participants’ minds. 

This framework was communicated as a broad set of principles for the program to fol- 
low, with room for elaboration and modification as the program operators gained experience. 
Unlike the financial incentives, which had been tested in Family Rewards 1.0, the family 
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guidance component of Family Rewards 2.0 had no model to follow, so this component of the 
program was more experimental. Even after the program launched in September 2011, several 
important aspects of the framework had to be developed by the lead program operator in 
collaboration with the NPOs and the Family Rewards Collaborative. As a result, this study of 
the implementation of family guidance describes how the philosophy and practice of guidance 
evolved over time, and does not attempt to measure fidelity to the foregoing list of guidance 
components. 

The next section describes the Advisors’ qualifications, prior experience with counsel- 
ing, and preparation for the role. 


Advisor Characteristics, Supervisory Structure, and Preparation 

Advisor Characteristics 

Most of the Advisors were recmited from other grants and projects within the NPOs. 
They were mostly young women with some postsecondary education. At BronxWorks, which 
had the youngest staff, all of the Advisors in the initial cohort were in their 20s and none had 
children. This set up a potentially challenging dynamic with parents in their caseloads, most of 
whom were around 40. At the same time, it afforded a unique opportunity for the Advisors to 
relate to the high school students. 

All of the Advisors had worked in social services, some in roles that included counsel- 
ing or case management. Since they were often coming from other programs at their NPOs, 
they tended to have deep knowledge of resources and systems related to those areas. Only two 
of the Advisors had special knowledge of the education or workforce domains. Five out of the 
six Advisors in the Bronx were bilingual in Spanish to accommodate the large number of adults 
there who preferred to communicate in Spanish. Across all NPOs, all but one of the Advisors 
reflected the populations they served in terms of ethnicity and race. 

Supervisory Structure 

At hiring, the Advisors were designated as “senior” or “junior” based on their educa- 
tional qualifications and years of experience. The expectation was that each NPO would hire 
two senior Advisors and one junior Advisor. All of the NPOs except CAS-NPO, in the Bronx, 
adopted this stmcture. At CAS-NPO, there was one senior Advisor and two junior Advisors. 
The senior Advisor obtained her master’s degree in social work during the program and had a 
formal role supervising her colleagues. At Memphis HOPE, one of the senior Advisors also had 
a master’s in social work and took on more supervisory responsibilities. At other NPOs, the 
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senior Advisors provided informal support to the junior Advisor by offering referral recommen- 
dations or helping in family guidance sessions, usually at the request of the junior Advisor. 

Each Bronx NPO served an average of 328 adults and 377 students, while each Mem- 
phis NPO served an average of 316 adults and 435 children. Each Advisor had a caseload of 
approximately 100 families, although at two NPOs the junior Advisors served fewer families 
than the senior Advisors. 

At each NPO the Advisors were overseen by a site supervisor who devoted between 10 
percent and 20 percent of his or her time to the project. As noted in Chapter 1, the site supervi- 
sors had a fair amount of influence over program design decisions and autonomy in managing 
their teams. For example, the Memphis NPOs decided to assign each Advisor to be a “champi- 
on” of one of the reward domains — education, health, or work. The Advisor responsible for a 
domain was expected to become an expert on community resources available to help families in 
that area, and to organize workshops or events to encourage families to earn rewards in that 
area. Advisors provided this service to all participants in addition to having primary responsibil- 
ity for their own caseloads. The supervisor at Porter-Eeath in Memphis also took the initiative to 
organize participants into teams during their initial orientation into the program. The idea was to 
generate a sense of community within teams and spark healthy competition among them. Each 
team was led by an Advisor and included all the members of his or her caseload. 

At the same time, the site supervisors received direction and technical assistance from 
the lead program operator’s program manager and program associates. In many cases, the 
program associates provided direct training to Advisors. The lead program operator (Seedco and 
then later CAS-Central) created performance indicators in collaboration with MDRC technical 
assistance staff members, and kept the site supervisors up to date on how their performance 
measured up to targeted outcomes. The main way Seedco, and then CAS-Central, provided 
performance feedback was by generating reports using data from the management information 
system (called Salesforce). In the first year of the program, these reports were focused on 
“process”: the number of participants who received orientation, the number of Family Earning 
Plans created, etc. In Year 2, CAS-Central developed a consistent definition of “low-earning 
participants” and began generating monthly reports that identified these individuals in each 
Advisor’s caseload. Advisors were expected to make contact with these participants and engage 
them in Family Earning Plan reviews with the aim of increasing their earnings. The focus of the 
performance feedback reports then shifted to the “outcome” of family guidance (that is, the 
reduction in the number of participants who had low earnings). 

In Year 2, three staff members at CAS-Central devoted at least some time to supporting 
the family guidance work. The program manager took the lead in drafting guidance materials, 
especially in Year 1. Two program associates were hired in Year 2, one in each city, who had a 
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background in social work and human resources development, respectively. They provided 
coaching on family guidance and the use of Salesforce, reviewed performance outcomes with 
site supervisors and staff members, and helped the NPOs develop and implement strategies to 
improve their performance. 

Preparation 

As described in Chapter 1, the first six months of the Family Rewards demonstration 
did not go according to plan. The delays in recmiting the sample had cascading effects on other 
elements of program development. This was exacerbated by Seedco’s very thin staffing 
stmcture, in which a single person was responsible for coordinating a response to recmitment 
challenges, leading the programming of Salesforce, training the Helpline staff, ensuring that 
payments were processed on time, developing program materials, and training staff members in 
their use. While the family guidance materials were designed carefully and the staff was trained 
in their use, there was not as much time as the program manager would have liked to monitor 
implementation closely. 

The early training sessions focused on the Family Earning Plan (Appendix Figure C.l) 
and a set of tools that staff members could use to assess participants’ barriers to earning rewards 
(for example, the “Work-Readiness Tool” shown in Appendix Figure C.3). The flow of the 
guidance session was expected to follow the stmcture of the documents. Seedco intentionally 
designed the documents in this way to support Advisors who had differing levels of experience 
with counseling. The Family Earning Plan was meant to walk an Advisor through marketing the 
incentives, eliciting a participant’s assessment of when he or she would begin to earn them, and 
discussing barriers to earning rewards and next steps. The assessment tools were developed 
soon after the Family Earning Plan and were intended to help staff members probe the underly- 
ing reasons participants might not earn rewards. The Advisors had a list of questions they could 
ask in each of the domains. For example, in the education domain, one assessment tool asked: 

• Do you have any responsibilities to take care of before you go to school in 
the morning? (For example, getting brothers and sisters up and ready, or car- 
ing for a grandparent or parent, or other adult in the house.) 

• How much time a night do you spend on homework? 

• How do you and your friends get along with other kids at school? 

These questions were expected to function like a decision tree guiding staff members to 
appropriate referrals. 

When Advisors described the messages they heard about their role, they recalled being 
firmly instmcted not to be “social workers” or to get too deeply involved in any participant’s 
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complicated life story. The style of advising was not meant to be intensive; it was meant to 
encourage participants to plan their own next steps rather than holding their hands. While this 
language is similar to what the Family Rewards Collaborative intended, the Advisors enacted 
their role in a way that was very different from the program designers’ expectation. A serious 
intervention was required at the start of the second year to bring the practice into better align- 
ment with the goals. 


The Evolution of Family Guidance 

Year 1 : Guidance Focused on Relationship Building and Customer 
Service 

According to Advisors and participants, two priorities shaped the character and content 
of family guidance in the first year of the program: 1) the need to build relationships with the 
participants, and 2) an emphasis on ensuring participants knew what activities had incentives 
attached to them and how to claim the rewards they had earned. 

Relationship Building 

Unlike other social programs that mandate meetings between clients and case manag- 
ers, Family Rewards made the relationship optional. ' For it to succeed, the participants had to 
view it as in their interest to attend the appointments. This was initially a challenge for Family 
Rewards because it took some time for the participants to start to earn rewards and build up a 
sense of frust in the program’s core offer. In all cases, participants had to go out of their way to 
visit their Advisors since the NPOs were not located in places participants typically frequented. 
One Advisor described this challenge: 

In the early phases of the program, we had to build a relationship with our prin- 
cipal participants. We didn’t know them, you know, these were just people that 
we signed up for the program. At first they were skeptical about the program: 

“Can 1 really get money for doing things we normally do with our families?” So 
once they started to see the money come in, it was a little different, but that first 
couple of months was kind of tricky in order to engage them and get them to 
come back and keep on going with it. 

The staff made persistent efforts to engage every family in person. In addition to Family 
Earning Plan review meetings, each NPO had an open-door policy allowing participants to drop 

'This was true in all but two cases. First, the reward for grades required that an Advisor sign the coupon in 
order for a student to be paid in Year 1. In Year 2, after a policy change allowed participants to mail in their 
report cards, students still had to meet with their Advisors once before any payments could be released. The 
reward for attaining a GED certificate also required an Advisor’s signature with verifying documentation. 
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by for help with payments or to talk about other issues. The factors most often cited by partici- 
pants in focus groups as contributing to strong relationships between Advisors and participants 
in Year 1 were: the report card reward; the Advisor’s persistent outreach; and the Advisor’s 
caring, nonjudgmental style. Lack of transportation was often cited as an impediment to a strong 
relationship, particularly in Memphis, as were any instances when an Advisor seemed unre- 
sponsive to the participant’s attempted outreach. In the end, transportation had a critical effect 
on the nature of guidance relationships in Memphis. 

In the first year of the program, the mles required that every report card be signed by an 
Advisor before payment could be released. In theory that mle gave the Advisor the opportunity 
to monitor each student’s academic performance and intervene with a referral or set of strategies 
if the student began to sttuggle. In Memphis, however, all high school students received their 
report cards on the same day. This created quarterly surges in the number of participants coming 
into the NPO to get their report cards signed. In New York City, every school could determine 
its own schedule for report cards so the flow was steadier. The spikes in visits to the NPO had 
negative consequences that are explained below, but the major positive consequence was that 
they brought participants into the NPO regularly. Students could earn rewards for Cs, so most 
students had at least one grade that qualified for a reward. The Advisors were very pleased to 
find that in the beginning some participants with no eligible grades brought their report cards in, 
thinking it was a program requirement. 

These frequent visits created familiarity between the Advisors and the participants in 
their caseloads. At one Memphis NPO where visitors had to sign into the building with their 
names, addresses, and phone numbers, the visits gave Advisors a way to stay up to date on 
participants’ contact information, which was important given that Memphis families turned out 
to be highly mobile. During this time participants were starting to earn rewards, so the credibil- 
ity and importance of the program grew in their estimation as they were becoming friendlier 
with the program’s staff 

The Advisors spent a significant amount of time on the phone reaching out to partici- 
pants. In Year I, many even called participants to let them know when payments had been 
posted to their accounts or to remind them about deadlines. These courtesy calls were not part 
of the triage model per se, but the participants who received these calls appreciated them. 
Though they were only required to meet with participants in their caseload twice a year, the 
Advisors attempted to be in regular contact with families, at least by phone. When a family did 
not respond to these efforts, the Advisors took extraordinary measures. At each NPO, Advisors 
described doing home visits, reaching out repeatedly by mail and phone, and searching for 
individuals on social media. 
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The adult participants described two qualities that made them open up to the Advisors. 
First, the Advisors seemed genuinely interested in the well-being of their entire families, and 
their concern seemed to go beyond just the program. Participants raved about Advisors who 
remembered their personal circumstances and made them feel valued. One mother in the Bronx 
described her Advisor as a part of her family because of her patience and sincerity: “That’s how 
you build good relationships with people, actually, not because it’s your job but because they 
take their job personal like you’re part of their family and they’re part of your family.” 

Students were surprised by their Advisors’ interest in their academic performance, and 

wanted to make them proud. The students in the Bronx who described close relationships with 

their Advisors also used analogies to family to describe their Advisors, saying she is like an aunt 

or somewhere between a parent and guidance counselor. However, for some students the fact 

that their Advisors were not family is what made the relationship meaningful. A female student 

in the Bronx said that her Advisor’s encouragement was special because “with my mom 

motivating me all the time it is like, okay, that’s your job, you’re supposed to tell me. But 

hearing it from someone else is just like, oh, they care too.” Many students in the Bronx 

2 

described traveling independently to the NPO to meet with their Advisors. 

These types of relationships were more characteristic of Advisor relationships in the 
Bronx, and of participants’ relationships with one Advisor in Memphis (who completed her 
master’s degree in special education before leaving the program). That Advisor was a 25-year 
veteran of social service programs in Memphis. Parents consistently praised her style. She 
would sit with participants and listen to their stories, sharing tips and referrals that came 
naturally to her due to her familiarity with Memphis social service agencies. An adult partici- 
pant described their conversations: 

Well, 1 come and get my coupons signed. We talk about jobs because 1 said, “1 
ain’t worked since 2011.” ... But anyway, she talk about having me calling eve- 
rybody, because she know everybody. She’s just real helpful. She’s very nice. 

We’d talk about my kids. She asked how my mama was doing and she’s just real 
concerned. Asked how my living arrangements is, if they’re going to school, is 
they getting in trouble. She’s very concerned. 1 love her. She’s the one that got 
me in the place where I’m living now. 

In addition, parents appreciated being able to ask questions about the program without 
being made to feel like a “nuisance.” This comment was often framed as a welcome contrast to 
typical relationships with case workers in social service programs. On the other hand, partici- 
pants took it very hard when they perceived that an Advisor was irritated by their questions, or 

^In Year 2 when the report card policy changed to allow students to mail in all grade coupons, some par- 
ents said they intentionally did not tell their children about the policy change so that the children would 
continue to visit the NPOs regularly and receive the extra encouragement. 
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when they called an Advisor and had to wait a long time to hear back. Complaints like these 
were few in the parent focus groups, but they came up mainly in Memphis. 

Parents and young people in Memphis generally said that they liked their Advisors but 
they did not, for the most part, have close relationships with them.^ They seemed to have very 
infrequent personal contact with their Advisors and to view them mainly as “reps” or “coordina- 
tors” they could call if they had trouble with their payments. The major reasons for this were the 
limited nature of the Advisor role (described below) and transportation obstacles. Because the 
participants relied on cars, which are expensive to operate, or rides from others, the Advisors 
had trouble getting participants to visit the NPO. The high school students relied on their parents 
for transportation so Advisors almost never met with high school students alone. Parents 
brought in students’ report cards to be signed. By the second year of the program, none of the 
staff members at one Memphis NPO could say that they had any regular personal contact with 
high school students in their caseloads. 

The Advisors in Memphis adapted to the persistent problem of participants not schedul- 
ing appointments or missing appointments due to transportation issues by spending much more 
time doing guidance on the phone, or by making home visits. While the Advisors were often 
ambivalent about bringing the program to participants, fearing that this would encourage 
passivity, they could not hit some performance targets without doing so. They believed that 
home visits could lead participants to invest more in the program and make arrangements to go 
to the NPO. The Advisors at Porter-Leath also adapted by hosting large group events. The 
events attracted participants with their promise of food and fun, and staff members used the 
occasion to make face-to-face contact with participants in their caseloads, and to complete some 
program-related business. 

Customer Service 

The Advisors believed that their role was mainly to ensure that participants understood 
their earning potential and the mechanics of claiming rewards, and to help resolve problems 
participants might have with the payment-processing system. In effect, they assumed a similar 
role as staff members in the Family Rewards 1.0 demonstration. “It was cut and dry,” one 
Advisor in Memphis explained. “If you came in, you came in for an issue” like signing a report 
card or turning in coupons. Another Advisor in the Bronx described the role as making sure 
“families are submitting for activities, regardless of how much money they’re receiving, just 
that they are submitting and they understand completely how much money the program has to 
offer.” 


should be noted that these focus groups took place in December 2012, after the staff was trained in mo- 
tivational interviewing, but before the technique had taken hold. 
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The Advisors did not explore barriers to earning rewards too deeply because they 
quickly found that participants who were willing to talk brought up complicated and wide- 
ranging personal challenges that the Advisors either felt proscribed from addressing effectively 
or that they did not know how to deal with. These issues included housing dislocation, HIV and 
other serious health problems, domestic violence, infidelity, divorce, substance abuse, teen 
pregnancy, and incarceration. Sometimes these barriers related directly to participants’ ability to 
earn rewards but often they were the set of forces that surrounded participants’ lives and that 
made an interaction focused exclusively on whether or not a person had earned a particular 
reward seem insensitive. The Advisors who had more case-management experience described 
an internal tension in which they wanted to explore these issues in greater depth and provide 
tme case management, which they understood as making phone calls for the participant and 
walking him or her through a plan of action. On the other hand, most Advisors acknowledged 
that they were not equipped to handle these kinds of problems and focused instead on being 
friends to the participants and developing expertise in the paperwork requirements for earning 
rewards. For these reasons, guidance sessions in Year 1 were both highly “transactional” (that 
is, focused on paperwork and program administration) and characterized by a lot of affirmation 
and requests for participants to commit to certain goals or by a more pointed, analytical style of 
reviewing earnings and missed opportunities. 

This affected the way the Family Earning Plan was administered. The focus groups 
found that participants generally understood the rewards, and the Advisors thought the Family 
Earning Plan helped them explain the rewards. It was an effective tool for reviewing each of the 
rewards and its value. But the other sections of the document were more difficult to implement 
as intended. First, the setting was not conducive to an honest or deep conversation with individ- 
ual participants. Entire families were brought in to complete the Family Earning Plan together. 
The parent Earning Plan reviewed workforce and health rewards, and each high school student 
completed his or her own student Earning Plan. With several other family members looking on, 
sometimes impatiently, the Advisors found that some participants (especially students) gave 
answers to the self-assessment section without much reflection, often responding that they 
would earn every reward immediately. Advisors also found it difficult to facilitate sensitive 
conversations about barriers to earning in this context. MDRC conducted a case file review at 
the end of Year 1 to look at hard copies of Family Earning Plans, and found that most had 
nothing written in the action-planning section. Staff members confirmed during focus groups 
that they tried to use the assessments but abandoned them because the questions were too 
difficult to ask or led into uncertain territory. 

In subsequent meetings with participants. Advisors developed their own style of inter- 
action. The following descriptions illustrate two styles that were seen in the Bronx and Mem- 
phis. These show the range of ways Advisors interacted with participants. 
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One style the researchers observed in guidance sessions with young people revolved 
around affirmation and bargaining. The Advisors congratulated students on earning rewards, 
encouraged them to earn more by telling them repeatedly, “You’re smart, you can do it,” and 
brought up things they could buy with the extra money if they succeeded. A student in the 
Bronx described what she took away from an advising session in which her Advisor used the 
affirmation style: 

When 1 meet with my Advisor ... she’s, like, helpful. She encourages me to take 
SATs, and she encourages me to keep doing better in school. Because at first I 
really didn't want to take — because 1 take my SATs in January and I don’t think 
I’m ready for them, but she motivated me to take them because she’s like, even 
if you don’t pass now ... it’s always okay to take it again. And she just tells me to 
not give up and don’t disappoint myself, because I’m going on a good roll, don’t 
ever, like, back down. 

If a student was having trouble in school, the Advisor would ask him or her to choose 
one or two classes and promise those grades would be higher at the next meeting. Much of the 
advising regarding grades was of this variety — or else the visit was strictly for the purpose of 
getting the Advisor’s signature. Because these frequent visits to the NPO were not generally 
leading to in-depth conversations about students’ academic needs, and because they required 
burdensome travel arrangements for families in Memphis, the signature requirement was 
removed at the start of Year 2. 

Still, many academically proficient high school students reported during focus groups 
that they believed they benefited from the encouragement they received from Advisors. The 
kind of encouragement Advisors gave them was more like mentorship than conventional 
academic counseling: the Advisors boosted students’ self-esteem, provided emotional support, 
and shared practical advice. While there is some evidence this kind of support can be benefi- 
cial,"^ it did not seem to the technical assistance staff at CAS -Central and MDRC to be adequate 
for students with serious academic challenges. 

The more analytical style of family guidance is represented by the following descrip- 
tion of a six-month Family Earning Plan review in Memphis. Advisors used a form in the six- 
month review meetings that helped participants visualize their earnings compared with the total 
offer (Appendix Figure C.2). The form called for Advisors to point out how much money 
participants had lost in the past six months by not meeting the conditions for rewards. The 
Advisors themselves had stumbled across this language through trial and error, and noticed that 
it grabbed people’s attention. The wording was also consistent with a psychological concept 


"^Jekielek, Moore, Hair, and Scarupa (2002). 
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called “loss aversion,” which states that people pay more attention to losses than gains.* Over 
time, however, the Advisors shifted away from this initial language and began instead to 
describe missed rewards as “unclaimed” or “left on the table.” They felt this approach stmck the 
same chord but was more empowering to participants since it suggested that they could still do 
something to earn it. 

An Advisor being interviewed during Year 2 of the program thought back to Year 1 and 
drew a contrast between the current and former styles. 

1 would start with the students. So maybe they’ve only earned $200, so they 
were missing $100 there. Is there anything we can offer? Well, we weren’t even 
saying that, 1 don’t think. We were just saying, “Okay, this is an area you might 
want to address,” because they’re leaving money on the table. 

That’s how you had to kind of word it or address it that way, like, “Okay, Miss 
Wilson, your child is leaving money on her table.” What 1 mean by that is they 
could have earned $300, but they only earned $200 in education. And then you 
have four kids here, plus yourself That’s five [people] in the household. Well, 
only two of your kids are paid dental, so you’re leaving some money as well. So, 
you might want to look at those areas. Then, you’ve got the work. “Well, Miss 
Wilson, are you working?” “No, I’m not working.” “Okay.” 

We hadn’t been given the green light to question why you weren’t working. 

Now if you shared that, you shared it, but it was nothing that 1 put in our notes or 
our [Family Earning Plan] or anything like that to say “Well, Miss Wilson said 
she’s not working because she has an addiction issue or she doesn’t have a GED 
or something like that.” It was like you don’t touch those issues. You leave it 
alone. 

While the two styles of advising from Year 1 are very different, they share some com- 
monalities: both are brief. Advisor-led conversations that compare the amount of money 
claimed with the maximum offer. 

Contrary to expectation, referrals did not generally play a large role in family guidance 
at this time. Advisors had not developed strong partnerships with other service providers so they 
were sometimes unsure of the options outside of their own organizations. Some Advisors also 
thought that giving referrals alienated participants at a time when Advisors were trying to build 
strong relationships with them. Advisors feared participants would feel guilty and stay away if 
they had not followed up on a referral. In Memphis, Advisors strongly (but mistakenly) be- 
lieved that they were not permitted to give referrals unless participants specifically requested 
them. 


*Kahneman and Tversky (1979). 


67 



The major exception to this pattern was found at CAS-NPO in the Bronx. From the 
start, this team used the site supervisor and an on-staff social worker to supplement the training 
provided by CAS-Central on how to discuss barriers to earning and provide support through 
referrals. 

The Family Resource Fund was not directly advertised to participants, making Advisors 
the gatekeepers for this money. The Fund was hardly used during the first year of the program. 
This seems to have been due to a combination of confusion about permissible uses; inability to 
use the funds to pay for dental visits, which is what many Memphis Advisors would have liked 
to do;*” the mismatch between the disclosure of an immediate need and the time it would take to 
get the funding; and a clear message from at least one site supervisor that it would be best to 
avoid using the Fund since there were often suitable free resources available in the community. 

Although CAS-Central had been providing half-day training sessions and other in- 
person coaching to Advisors during the first year of the program, the program manager and 
other members of the staff recognized that a more coordinated and intensive approach would be 
needed to build the Advisors’ skills. The main goal was to bring greater uniformity to the style 
of advising and improve Advisors’ ability to explore barriers to earning now that they had 
established a strong foundation of tmst with participants and helped them understand the 
rewards. This was seen as essential to moving participants from claiming rewards they had 
earned to earning rewards in new areas. 

Year 2: Guidance Focused on Referrals and Motivational Interviewing 

Year 2 of the program can be divided into two periods with respect to family guidance. 
In the first six months, the NPOs developed stronger ties to referral partners and CAS-Central 
introduced a new approach to family guidance based on motivational interviewing. During the 
second six months, motivational interviewing had taken root and staff members applied it fairly 
consistently to their Family Earning Plan reviews. 

New Referral Partners 

In a process that began at the end of Year I, the NPOs, with guidance from CAS- 
Central, developed partnerships with new referral partners. In Memphis, these were providers 
who offered low-cost dental cleaning to adults with no insurance. In the Bronx, these were 
workforce programs. These referral networks were described in Chapter 2. 


^This policy was changed in the third year of the program to address the persistent challenge of finding 
dental providers in Memphis. 
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The Introduction of Motivational Interviewing 

After considering multiple factors (including the relatively brief duration and sporadic 
frequency of guidance sessions and the uneven skill levels of staff members), CAS-Central 
decided to bring in a new approach to family guidance at the start of Year 2 organized around 
motivational interviewing. The purpose of this change was to give Advisors the tools to engage 
low-earning participants, and to stmcture effective conversations about behavior change, rather 
than simply helping participants claim rewards they had earned. 

Motivational interviewing is, according to one definition, “a treatment philosophy and a 
set of methods employed to help people increase intrinsic motivation by exploring and resolving 
ambivalence about behavioral change.”’ Developed in the 1980s in the context of substance- 
abuse treatment, it has exploded in popularity in the 2000s in numerous counseling settings, 
although most of its evaluated uses still relate to addiction and health promotion. It is a client- 
driven approach to counseling in which individuals are regarded as the experts in improving 
their own lives. This does not mean that the counselor is passive. To the contrary, motivational 
interviewing is directive, in that the counselor uses a set of techniques to explore the individu- 
al’s reluctance to change. The ultimate goal is for the individual to express a verbal commitment 
to change (“change talk”). 

Motivational interviewing has been found to be more effective than no treatment, and at 
least as effective as some psychotherapeutic alternatives like cognitive behavioral therapy that 
take more time to implement.* It is a flexible set of techniques that can be incorporated into 
existing programs, and has been found to increase program engagement especially for members 
of certain racial and ethnic minority groups.® For Family Rewards, the major risks involved in 
adopting it were: staff members’ resistance to this significant shift in the nature of their jobs; the 
difficulty of achieving a minimum level of staff competence in it given the range of skill levels 
Advisors had; and the lack of evidence regarding its effectiveness in nonclinical settings where 
participants do not necessarily see their lives as in need of change. 

CAS-Central introduced the Family Rewards staff to motivational interviewing during a 
two-and-a-half-day retreat in Memphis in early November 2012. A certified motivational 
interviewing trainer facilitated the workshop, which involved a combination of lectures and 
role-playing. NPO site supervisors embraced the new approach, but the initial reaction among 
Advisors was mixed. Motivational interviewing was a major change for them. As one Advisor 
put it, it was going from “close-ended statements to open-ended questions.” Five of the 12 
Advisors were enthusiastic supporters right away, while at least 4 had strong reservations. 

’Miller and Rollnick (2002) as quoted in Lundahl and Burke (2009), p. 1,232. 

*Lundahl and Burke (2009). 

®Hettema, Steele, and Miller (2005). 
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Advisors in the latter group had felt confident in their ability to provide customer service and 
referrals (once the new referral networks were in place), but felt unprepared for this new 
approach. Most Advisors described feeling overwhelmed by the expectation that they would be 
able to implement the practice right away, since it had so many components. They wondered 
whether every interaction with participants had to be infused with motivational interviewing, 
and a few stmggled to find a way to sound natural while using the motivational interviewing 
language they had been taught. 

CAS-Central invested significant resources in providing ongoing training for staff 
members, recognizing that Advisors would need a lot of in-service support to develop their 
skills. Two program associates were hired in Year 2 whose primary responsibility was staff 
professional development. They created tip sheets and tools to use during guidance sessions. 
The Advisors began tape-recording their guidance sessions and handing those tapes to their site 
supervisors and their program associates. The recordings were reviewed and scored according 
to the number of motivational interviewing techniques the Advisor used, and the degree to 
which the Advisor capitalized on opportunities to advance the conversation toward change talk. 
Every other week program associates facilitated case conferences attended by all Advisors at an 
NPO and the site supervisor, where they played a recorded guidance session and discussed it. 
The Advisors described case conferencing as an effective method for continuing to improve 
their skills since they were able to hear their peers applying motivational interviewing and 
discuss strategies collectively. The case conferences also represented opportunities to review the 
team’s progress in relation to performance metrics, and to provide short training sessions. 

The first six months of Year 2 was a period of tremendous variation in the amount and 
quality of motivational interviewing that programs used. While a few Advisors took to the 
methods immediately, the majority needed more training or a direct order from their site 
supervisors to begin to try out the techniques, as described in Box 3.1. 

Full Implementation of Motivational Interviewing 

According to CAS-Central and Advisors themselves, all staff members achieved a min- 
imum level of proficiency in motivational interviewing by the end of the summer of 2013, with 
many of them becoming very good at it earlier. The length of time it took to achieve minimum 
competency demonstrates the significant organizational investment in professional development 
that was required to make the transition to motivational interviewing possible. 

The texture of guidance sessions changed after the full adoption of motivational inter- 
viewing. Most of the Advisors eventually bought into the approach and saw it as beneficial. 
They described breakthroughs with participants, particularly in eliciting engagement among 
uninterested participants or in getting participants to commit to new goals. There was no 
“control” group condition in which participants were offered the incentives and a different style 
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SIF Family Rewards 
Box 3.1 

Motivational Interviewing in the Service of the Financial Incentives or the 

Other Way Around? 

A Perspective from the Family Rewards Program Manager at CAS-Central 

liana Zimmerman 

Before motivational interviewing training, Advisors encouraged participants to set short-term goals 
to claim rewards. Discussion tended to focus on rewards for which a participant was already 
demonstrating the target behavior but needed to compile paperwork for the program’s verification, 
or activities in which a participant felt invested but needed an additional push or reminder. 

Once motivational interviewing was implemented. Advisors’ focus shifted to participants’ long-term 
goals. Suspending judgment. Advisors talked to participants about their current behavior and 
lifestyle choices. They used motivational interviewing to develop discrepancy: highlighting the gap 
between where participants hoped to go and where their current behavior was leading them. As they 
became aware of the gap, participants started to explore alternative behaviors that would better 
support their goals, and for the first time many expressed co mmi tment to attempting Family 
Rewards activities. 

Although it may seem that attention shifted away from earning rewards, just the opposite is hue. 
Take the following case. 

An Advisor worked with a mother who could not articulate any goals related to 
health, work, or her child’s education. She was preoccupied with her housing situa- 
tion. Her family lived in a small Bronx apartment with her parents, and she wanted 
to find her own place for herself and her children. She and her Advisor began ex- 
ploring the fit between her goal of finding an apartment and her current behaviors, 
particularly her decision not to work. As the guidance session unfolded, the mother 
came to the conclusion that employment was the only way that she could achieve 
her goal. For the first time since starting Family Rewards, she expressed a com- 
mitment to finding a job. She followed through, and a few months later started 
work in a medical facility. 

Motivational interviewing led more participants who at first did not intrinsically desire to work full 
time or to maintain good attendance in school to see these activities as steps to achieving their future 
goals. Fa mi lies became more likely to believe that the new habits they practiced and the cash they 
earned would benefit them now and in their future. 
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of advising or no advising at all, however, so it is impossible to verify these accounts empirical- 
ly. The full implementation of motivational interviewing coincided with the full implementation 
of targeted outreach to low-earning participants as well as the availability of new referral 
options, making it difficult to tease out the specific contribution of motivational interviewing 
from that of these other program developments. The following observation of an Advisor in the 
Bronx who was highly proficient in motivational interviewing illustrates how nuanced it could 
make conversations with participants. 

The Advisor asked the student about her English class. “What have you been do- 
ing different in English that raised your grade?” The student said she had been 
participating more in class. Advisor asked, “What else could you do?” The stu- 
dent explained that the class asks for a reading report, but it is boring. She reads 
the material, but does not like writing the stmctured, five-part reports. The Advi- 
sor noted that the participant was already doing half of the work by reading the 
material and that she had seen that if she put in the effort, it improved her grades. 

She suggested that maybe she could go halfway: instead of completing the five 
parts of the reading report, she could start with two. The participant seemed to 
like the idea.... 

Then the Advisor went over the “goal-setting tool” and asked the participant to 
write down four goals [Appendix Figure C.4]. Afterward the Advisor read each 
of the goals, asked the participant how confident she was about reaching her 
goal, and then asked some follow-up questions. For example, where the student 
wrote, “Do all “Global” homework — 9 confidence level,” the Advisor asked, 

“Why 9 and not 2?” The student responded that she had free time and knew she 
could do it. 

After going through the list of goals, the Advisor suggested that the participant 
place the sheet in a place where she could see it so she would remember it. She 
said they would go over it the next time they met. 

Most Advisors said that participants, especially students, adapted well to the change in 
the guidance style. In addition, motivational interviewing gave Advisors new tools to confront 
parents’ ambivalent feelings about full-time work. As noted in Chapter 2, parents questioned 
whether they would be better off working, worrying about their younger children and the loss of 
public benefits. Advisors used motivational interviewing techniques to get participants to 
explore their own resistance, while trying not to push too hard. They were primarily interested 
in keeping the lines of communication open in the hope that a parent would express a readiness 
to work in the future, at which time the Advisor could provide more intensive support. 

CAS-Central provided an example of a sophisticated guidance in relation to the work- 
force coupon. A Memphis Advisor asked a parent about work and the parent said she was out of 
work but was thinking about returning to her old job. The parent said she did not know if she 
really wanted to return to work, but she did know she wanted to go back to school. The Advisor 
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used a technique called a “double-sided reflection” to explore this uncertainty: “On the one 
hand you want to go back to work to get money but on the other hand you really want to go to 
school.” The Advisor then asked a “looking forward” question: “What does your life look like 
in five years?” In response, the parent shared that she wanted to start her own business. Then, 
the Advisor asked, “What does it look like financially if you stay where you are?” The parent 
replied that she really could work and go to school at the same time. At this point in the discus- 
sion, the Advisor summarized their conversation and asked the parent what stood in the way of 
going back to work. The parent described not having a sitter, and loss of benefits. The Advisor 
stated that she would like to meet again with the parent to go over the “change plan worksheef ’ 
to map out next steps and resources needed to reach the parent’s goal. The parent agreed. 

CAS-Central trained staff members to follow a set stmcture in conducting guidance 
sessions using motivational interviewing so there was some continuity between guidance 
sessions conducted by different Advisors (although the above examples demonstrate how fluid 
conversations could be). Advisors were trained to prepare for the meeting before the participant 
arrived by reviewing Salesforce data on earnings and notes from prior meetings. When the 
participant arrived, they would set the agenda together. The bulk of the meeting would be 
devoted to exploring unclaimed reward opportunities and setting goals for earning using one of 
a variety of motivational interviewing tools that CAS-Central provided. When requested by the 
participant, the Advisor could provide referral information. The Advisor would then wrap up 
the session by summarizing what had been accomplished and expressing confidence in the 
participant’s ability to achieve his or her goals. The Advisor and the participant might also make 
a plan for following up with each other, which could be initiated by either party. After the 
participant left, the Advisor would document the session by writing case notes in Salesforce. 

Continuing Challenges with Motivational Interviewing 

While motivational interviewing led to many apparent improvements in family guid- 
ance when implemented well, staff members described three continuing challenges after it was 
introduced. 

First, Advisors with weaker motivational interviewing skills stmggled to reorient them- 
selves from the customer-service approach to guidance they had adopted in Year 1. Not all 
participants viewed their Advisors as someone they could turn to for help earning new rewards 
or dealing with other life problems that stood in the way of earning rewards. The inability of 
some Advisors to draw participants into deeper conversations meant that these Advisors had 
fewer opportunities to practice and improve their motivational interviewing skills. The weaker 
Advisors found that conducting guidance sessions in tandem with more proficient Advisors 
helped address this problem. Each of the NPOs took a team approach to family guidance, with 
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Advisors often seeing each other’s cases when one was unavailable or thought a colleague 
could provide better assistance, so conducting some guidance sessions together was natural. 

Second, since it was not mandatory for participants to come into the NPO for meetings 
with Advisors after the report card rule change, many did not, meaning Advisors had to conduct 
biannual review sessions by phone. Phone guidance was more common in Memphis, where one 
Advisor estimated that 60 percent of her biannual meetings had been conducted by phone, not 
the optimal context for having meaningful conversations with participants. The Advisor 
explained: 

It does kind of put up a wall because you know, you don’t know what they are 
doing on the other end. They could be watching TV, or not paying attention.... 

So, it definitely is a little barrier. And it’s a little weird too: “I’m calling to dis- 
cuss your earnings.” Like okay. What does that mean? And 1 haven’t talked to 
you in six months. 

Last, Advisors tended to believe that they had to incorporate motivational interviewing 
into every encounter with participants and could never deviate from its particular style of 
communication. For example, they questioned whether they could make suggestions or add 
goals to a participant’s list (especially in cases where the participant was being targeted because 
of low earnings and the Advisor was under pressure to get the participant to focus on raising 
earnings in that area). CAS-Central attempted to reassure Advisors that they should be using a 
variety of styles of guidance including more prescriptive or customer-service-oriented styles 
when appropriate. The key was to find the right moments for different styles of communication 
and adhere to the overarching philosophy of motivational interviewing — collaboration, 
empathy, and change — at all times. 


The Amount and Distribution of Guidance 

The MDRC research team acquired all of the case-management data from Salesforce, the 
management information system that CAS-Central developed for the program. This data source 
provides a comprehensive overview of the scope, intensity, and distribution of the interactions 
between Advisors and participants, but it is limited by the way the Advisors interpreted their 
own work, their consistency in applying codes, and the numerous upgrades to the system that 
were made over time. CAS-Central spent a lot of time training staff members in the proper use 
of Salesforce, and in focus groups Advisors did claim that they were using it consistently by the 
start of Year 2. 

The research team categorized interactions with participants based on the intensity of 
services delivered: high-intensity guidance, medium-intensity customer service, and low- 
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intensity outreach.'® Guidance sessions are interactions that explore participants’ accomplish- 
ments and attempt to address needs. Customer-service interactions are largely transactional 
meetings concerning payments. Outreach consists mostly of phone calls and letters to schedule 
meetings or market the rewards. Participants may or may not have actually received these letters 
and phone calls. 

In reviewing these data, it is important to note the difference in workload between the 
Advisors in the Bronx and Memphis. Because Memphis families were larger, Bronx NPOs had 
91 fewer participants overall (see Table 3.1).The largest discrepancy was between CAS-NPO in 
the Bronx and Memphis HOPE: CAS-NPO had the smallest caseload with 678 individuals, 365 
of whom were students, while Memphis HOPE had the largest with 773 individuals, 450 of 
whom were students. 

Creating and Reviewing Famiiy Earning Pians 

As noted earlier, the creation of the Family Earning Plan was the first one-on-one inter- 
action between an Advisor and a participant, and it served to educate participants about the 
incentives and motivate them to earn. The Salesforce data reveal that Family Earning Plans 
were created for most participants (Table 3.2). About 94 percent of adults across both cities had 
created a Family Earning Plan by the end of Year 2. For high school students, the percentage 
was lower — 90 percent in the Bronx and 77 percent in Memphis. This discrepancy reflects the 
difficulty Memphis HOPE Advisors had bringing students in for this activity. At that NPO, only 
69 percent of high school students had completed this activity by the end of the second year 
(Appendix Table C.l). Memphis HOPE Advisors attributed this to the particularly disadvan- 
taged circumstances of families in their caseloads and the fact that their organization had the 


*®Each of these categories corresponds to a set of fields Advisors could select in Salesforce. Multiple 
codes could be selected for a single interaction. In this analysis, where there are multiple codes recorded for a 
single guidance session each of them is included in the count for its type of interaction, but the number of 
guidance sessions is counted by the date of the occurrence to avoid overstating the number of interactions. 

• High-intensity guidance includes the codes: Family Earning Plan review, Family Earning Plan creat- 
ed, Year 2 report card review for students, discussed next steps, discussed barriers, discussed public 
benefits, and referral made. 

• Medium-intensity customer service includes the codes: group check-in, group workshop, discussed 
referral outcome, administrative contact (submitting bank account information or adjusting family 
composition or eligibility), and program customer service (Advisor signature for GED certificate 
completion. Advisor signature for report card for students, answered earnings questions. Helpline call, 
or coupon submission support). 

• Low-intensity outreach includes the codes: program campaign (outreach to particular participants 
with information or reminders), scheduled coupon book pick-up, program reengagement call, and 
program outreach (sent e-mail, sent letter, or conducted automated call). This category includes forms 
of outreach where the Advisor did not actually make contact with the participant. 
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SIF Family Rewards 
Table 3.1 


Caseload Size, by Neighborhood Partner Organization 


Outcome 

Bronx Works 

CAS- 

NPO 

Memphis 

Hope 

Porter- 

Leath 

Adults 

342 

313 

323 

308 

Average number of adults per staff member 

114 

104 

108 

103 

Students 

389 

365 

450 

419 

Average number of students per staff member 

130 

122 

150 

140 

All participants 

731 

678 

773 

727 

Average number of participants per staff member 

244 

226 

258 

242 


SOURCE: MDRC calculations using baseline data. 


NOTE: Among children, only participants in high school were eligible for guidance. Only students 
identified as being in high school in Year 2 are included. 


most students to serve. Students who did not have this meeting missed an opportunity to 
become familiar with the program and their Advisors, which may have affected their engage- 
ment with the program. 

The Family Earning Plans were not reviewed twice yearly, as anticipated by the origi- 
nal family guidance model. Figure 3.1 shows that 52 percent of adults in the Bronx and 40 
percent in Memphis attended one Family Earning Plan review meeting in Year 2. A smaller 
percentage attended two review meetings: 30 percent in the Bronx and 21 percent in Memphis. 
Among high school students, the percentages were even lower. Forty-nine percent of students in 
the Bronx and 24 percent in Memphis attended one Family Earning Plan review; 24 percent of 
Bronx students came back for a second meeting, while only 2 percent of Memphis students 
did." This means that relatively few participants received formal biannual guidance. Advisors 
in Memphis in particular noted a significant decline in participants’ willingness to attend the 
meetings once they no longer had to do so to claim the grade reward. NPOs’ inability to bring 
most participants back for multiple meetings demonstrates one of the challenges of the 

"One of the adaptations Advisors instituted was to conduct student Earning Plan reviews with parents. 
CAS-Central endorsed this approach. If an Advisor completed a Family Earning Plan review with a parent, 
therefore, that Advisor sometimes did not try to do a separate one with the student in that family. These 
numbers do not indicate how many families had one or two Family Earning Plan reviews for either the adult or 
a high school student. 
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SIF Family Rewards 
Table 3.2 

Family Earning Plans Created by tbe End of Year 2, by City 


Outcome (%) 

Bronx 

Memphis 

Total 

Adults 




Created Earning Plan 

94.5 

92.7 

93.6 

Sample size 

655 

631 

1,286 

Students'* 




Created Earning Plan 

90.3 

77.2 

83.3 

Sample size 

754 

869 

1,623 


SOURCE: MDRC calculations using CAS's Family Rewards program data. 

NOTE: ^Sample only includes students who were identified as being enrolled in high school 
during Year 2. 


voluntary nature of family guidance. Because guidance in the first program year and at least half 
of the second consisted of mostly transactional interactions, participants may not have felt that it 
was worth it to attend or make themselves available by phone more than once, even as the 
Advisors were learning motivational interviewing and trying to make the interaction more 
meaningful. 

One NPO was able to maintain a higher level of contact with students over time. Ap- 
pendix Figure C.5 shows that BronxWorks Advisors conducted two Family Earning Plan 
reviews with 35 percent of the organization’s students, whereas at other NPOs the proportion of 
students who had a second review ranged from 13 percent to 2 percent. More students attended 
a second Family Earning Plan review at BronxWorks because when the program’s mles 
changed. Advisors did not tell students that they no longer needed to attend these meetings to 
claim their grade rewards. 

While Family Earning Plan reviews were important check-ins, they must be understood 
in the context of all of the forms of interaction between Advisors and participants. According to 
the classification system described in the introduction to this section. Advisor-participant 
interactions included customer service to address payment issues and outreach to provide 
information or simply reconnect. At times Advisors would provide participants with elements 
of a guidance session — by discussing next steps or barriers to earning, or by providing referrals 
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SIF Family Rewards 
Figure 3,1 

Summary of Family Earning Plan Reviews in Year 2, by Participants and City 


Adult participants 


Student participants 
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■ Completed 1 review □ Completed 2 reviews □ Completed more than 2 reviews 


SOURCE: MDRC calculations using CAS's Family Rewards program data. 

NOTE: Year 2 covers activities that occurred between September 2012 and August 2013. Sample includes students 
who were identified as being enrolled in high school during Year 2. 

— outside of a formal Family Earning Plan review. Table 3.3 shows that almost every adult in 
the program experienced a Family Earning Plan review or elements of a guidance session (80 
percent), customer service (71 percent), and outreach (91 percent). Similarly, almost all students 
had contact with their Advisors at least once, most through a guidance session (85 percent) or 
customer service (66 percent), and 43 percent through general outreach. The differences 
between the Bronx and Memphis in this regard are minimal, except that a much smaller 
percentage of high school students in Memphis received general outreach. 
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SIF Family Rewards 
Table 3.3 


Summary of Guidance Sessions in Year 2, by Participant and City 


Outcome (%) 

Bronx 

Memphis 

Total 

Adult uarticiuants 




Participant engaged in at least 1 interaction 

97.1 

97.0 

97.1 

Guidance"* 

87.3 

72.4 

80.0 

Customer service** 

69.5 

72.6 

71.0 

Outreach** 

87.9 

93.7 

90.8 

Sample size 

655 

631 

1,286 

Student uartiduants'' 




Participant engaged in at least 1 interaction 

92.4 

90.0 

91.1 

Guidance"*’** 

85.8 

83.3 

84.5 

Customer service**’^ 

69.5 

63.6 

66.4 

Outreach** 

65.8 

24.1 

43.4 

Sample size 

754 

869 

1,623 


SOURCE: MDRC calculations using CAS's Family Rewards program data. 

NOTES: Year 2 covers activities that occurred between September 2012 and August 2013. 

“Guidance includes: creating Family Earning Plan, reviewing Family Earning Plan, 
discussing next steps, discussing barriers to earning, discussing public benefits, or making 
a referral. 

'’Customer service includes: group check-in, group workshop, discussing referral 
outcome, submitting bank account information, adjusting family composition/eligibility. 
Advisor signature for GED certificate completion, answering earnings questions. Helpline 
call, and coupon submission support. 

‘’Outreach includes: program campaign (outreach to particular participants with 
information or reminders), scheduling coupon book pick-up, program reengagement call, 
sending e-mail, sending letter, and conducting automated call. 

“^Sample only includes students who were identified as being enrolled in high school 
during Year 2. 

‘’Category also includes report card review for students. 

^Category also includes Advisor signature for student report cards. 
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Targeted Outreach to Help Participants Earning Few Rewards 

This analysis regards the most intensive Family Earning Plan reviews as those that in- 
cluded discussions of barriers to earning rewards or next steps, or that included a referral. Each 
of these elements deepens the conversation beyond the transactional interaction that character- 
ized Family Earning Plan reviews in Year 1. The combined effect of the implementation of 
motivational interviewing, new referral partners, and the “targeted outreach” that went into 
effect around Febmary 2013 should have been to increase the intensity of guidance sessions in 
the second half of Year 2 and to concentrate those guidance sessions on low-earning partici- 
pants. Given the size of Advisors’ caseloads and the fact that it is more challenging to connect 
with participants who are not earning many rewards, the triage model assumed that Advisors 
would transfer their focus from participants who could earn rewards on their own to those who 
were harder to serve. 

CAS-Central created a definition of “low-earning” in the middle of Year 2 to focus Ad- 
visors on this subgroup of adults and students. “Eow-eaming” adults were individuals who had 
earned health or workforce rewards in Year 1 but not Year 2.'^ “Eow-eaming” students were 
individuals who had not earned at least $75 for grades. These definitions did not coincide 
completely with the adult and student subgroups that the program designers had hoped this 
program model would serve better than Family Rewards 1.0, that is, adults without a high 
school diploma or GED certificate and nonproficient high school students. Figure 3.2 shows 
that only 117 adults in the Bronx and Memphis (or 9 percent of the total adult sample) were 
both “low-earning,” according to this definition, and lacking a high school diploma or GED 
certificate. A somewhat greater proportion of low-earning students were also nonproficient: 490 
students (or about 41 percent of the total student sample) were both low-earning by Year 2 and 
nonproficient in English at the start of the study. Yet overall, it is clear that by targeting partici- 
pants based on these particular definitions of low reward earnings. Advisors were not substan- 
tially reaching the subgroups whom Family Rewards 1.0 found to be at risk of not earning 
rewards. If the impact analysis for Family Rewards 2.0 again finds negative or no impacts in the 
workforce and education domains for these subgroups, this mismatch may be a contributing 


*^Note that adults who did not earn rewards in e/tAer program year are not included in this definition. 
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factor, but CAS-Central and the NPOs did not receive a clear directive from the Family Re- 
wards Collaborative to structure their targeted outreach strategy differently.'^ 

Figures 3.3 and 3.4 illustrate that, in line with predictions, the intensity of guidance ses- 
sions generally increased after the NPOs began the targeted outreach strategy, represented by 
“Period 2” in the figures. In addition, the figures show that participants who would be identified 
as low-earning at midyear received more high-intensity guidance sessions in the second half of 
Year 2 (Period 2) than they did in the first six months of Year 2 (Period 1). For students in both 
cities, but especially for those in Memphis, the focus on low-earning participants resulted in a 
shift away from students who had more than $75 in rewards from grades. The low-earning 
adults in both cities received more guidance sessions and guidance sessions of higher intensity 
than they had previously, but higher-earning adults received the same amount of guidance 
services as a group (Memphis) or more (Bronx). Thus, the triage model was implemented well 
for students, less so for adults. 

Although the intensity of interactions between Advisors and participants did increase, it 
is worth questioning whether the interactions were intensive enough to address participants’ 
substantial needs. Contrary to the expectations of the program designers, referring participants 
to other services within the NPOs or in the community was not a significant part of the Advi- 
sors’ practice, either in the context of Family Earning Plan reviews or in any other interaction. 
For example, in the first year of the program, only 8 percent of adults and 1 percent of students 
received an Earning Plan review that included a referral (Table 3.4). In Year 2, the proportions 
of participants receiving Family Earning Plan reviews and the level of intensity of those reviews 
increased, but the percentages receiving referrals were virtually unchanged: 9 percent of adults 
and about 1 percent of students received an Earning Plan review that included a referral. 


'^Appendix Tables B.2 and B.3 show the earnings in the workforce and education domains for adults and 
students in the subgroups discussed in Figure 3.1. These figures provide a preliminary look at whether adults 
without a high school diploma or GED certificate and students who were nonproficient in English are lagging 
behind their counterparts in earning workforce and education rewards. Appendix Table B.3 shows that 41 
percent of Bronx adults without a high school diploma or GED certificate earned the workforce reward at least 
once, compared with 47 percent of Bronx adults with more education. The corresponding difference between 
these adult subgroups is larger in Memphis. Appendix Table B.2 shows that students who were not proficient 
in English at the start of the program earned the grades coupon at a lower rate than students who were 
academically proficient, but the fraction of nonproficient students who earned this reward at least once is still 
high, at 75 percent in the Bronx and 81 percent in Memphis. The largest difference between student subgroups 
is with respect to the state exam, where only 45 percent of nonproficient Memphis students earned a reward for 
the End of Course exam compared with 85 percent of proficient students. In sum, these tables show a persistent 
difference in earnings between adult subgroups based on education level and student subgroups based on 
academic proficiency, but the percentages who have earned workforce and grade rewards, respectively, are 
fairly close. The real question is whether earnings for grade rewards is a sufficient indicator of engagement or 
academic performance to affect program impacts given the relatively low percentage of nonproficient students 
who earned the reward for passing an End of Course exam in Memphis. 


81 



SIF Family Rewards 
Figure 3,2 

Low-Earning Participants and Baseline Characteristics 

Adult participants 

Sample size = 1,270 

No high school diploma or GED 

Low-earning certificate at the time of enrollment 

Sample size = 306 Sample size =510 



Student participants 

Sample size = 1,204 

Nonproficient in English 
Low-earning at the time of enrollment^ 

Sample size = 564 Sample size = 982 



SOURCE: MDRC calculations using CAS's Family Rewards program data and baseline 
survey data. 

NOTES: Sample sizes may vary due to missing values. 

^Proficiency level is only reported for high school students who had taken a 
standardized test to determine proficiency within the two years prior to enrollment. Data 
were available for most students who were in ninth or tenth grade at the time of 
enrollment. 
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Number of guidance sessions 


SIF Family Rewards 
Figure 3.3 


Adult Guidance Sessions in Year 2, by Subgroup and City 



Bronx Memphis 

□ Did not include discussion of next steps, discussion of barriers to earning, or referral 
■ Included discussion of next steps, discussion of barriers to earning, or referral 


SOURCE: MDRC calculations using CAS's Family Rewards program data. 

NOTES: Year 2 covers activities that occurred between September 2012 and August 2013. Period 1 includes 
guidance sessions between September 2012 and February 2013. Period 2 includes guidance sessions between 
March 2013 and August 2013. 

Guidance includes: Creating Family Earning Plan, reviewing Family Faming Plan, discussing next steps, 
discussing barriers to earning, discussing public benefits, or making a referral. 
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Number of guidance sessions 


SIF Family Rewards 
Figure 3,4 

Student Guidance Sessions in Year 2, by Subgroup and City 



Low-earning Higher-earning Low-earning Higher-earning 

Bronx Memphis 


□ Did not include discussion of next steps, discussion of barriers to earning, or referral 
■ Included discussion of next steps, discussion of barriers to earning, or referral 

SOURCE: MDRC calculations using CAS's Family Rewards program data. 

NOTES: Year 2 covers activities that occurred between September 2012 and August 2013. Period 1 includes 
guidance sessions between September 2012 and February 2013. Period 2 includes guidance sessions between 
March 2013 and August 2013. 

Guidance includes: creating Family Earning Plan, reviewing Family Earning Plan, discussing next steps, 
discussing barriers to earning, discussing public benefits, making a referral, or reviewing a Year 2 report card. 
Sample only includes students who were identified as being enrolled in high school during Year 2. 
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SIF Family Rewards 
Table 3.4 


Participants Who Received Intensive Guidance During Earning Plan Review, 

by Year and City 




Year 1 


Year 2 

Outcome (%) 

Bronx 

Memphis 

Total 

Bronx 

Memphis 

Total 

Adults 







1 or more review meetings included discussion of next 
steps, discussion of barriers to earning, or a referral 

22.9 

3.7 

13.5 

30.5 

22.2 

26.4 

At least 1 included a referral 

12.1 

3.5 

7.9 

6.3 

12.0 

9.1 

Sample size 

655 

631 

1,286 

655 

631 

1,286 

Students" 







1 or more review meetings included discussion of next 
steps, discussion of barriers to earning, or a referral 

11.5 

0.7 

5.8 

22.9 

6.6 

14.4 

At least 1 included a referral 

1.4 

0.5 

1.0 

0.6 

0.8 

0.7 

Sample size 

695 

760 

1,455 

695 

760 

1,455 


SOURCE: MDRC calculations using CAS's Family Rewards program data. 


NOTES: Year 1 covers activities that occurred between September 2011 and August 2012. Year 2 covers activities 
that occurred between September 2012 and August 2013. 

“Includes students in high school in Year 1 and Year 2. 



Conclusion 

CAS-Central and the Advisors approached the development and execution of the family 
guidance component with an intense commitment to continuous program improvement. In the 
first year, advising did not fully reflect the triage model since Advisors did not explicitly target 
low-earning participants, nor were they able to move conversations into a discussion of barriers 
to earning rewards. Instead, Advisors aimed to build friendly, tmsting relationships with 
participants and ensure they were able to handle the program mechanics necessary to get paid 
for the activities they completed. These achievements laid the foundation for the elaborations to 
the guidance model that were implemented in Year 2, which included adding greater depth to 
the advising approach through motivational interviewing, creating new referral partnerships, 
and targeting low-earning participants for outreach. Unfortunately, most participants did not 
have the anticipated two formal guidance sessions in Year 2 after motivational interviewing was 
introduced, so although the quality of guidance does appear to have improved, the amount of 
guidance most participants received was relatively low. 

One of the greatest surprises of the family guidance component is the relatively small 
number of referrals Advisors have made. At the beginning of the program, it was anticipated 
that Advisors would function as “resource experts” who developed knowledge of the services 
available in their communities and connected participants to appropriate agencies. This was 
seen as the alternative to Advisors providing services directly. Although the number of referrals 
for dental services increased dramatically in Memphis in Year 2, the overall proportion of 
guidance sessions that included referrals was still low, suggesting that the guidance Advisors 
provided was focused more on using communication strategies to elicit an intrinsic commitment 
to change than on formal referrals. That is, the guidance interaction targeted motivation and 
patterns of belief more than the program designers anticipated. It remains to be seen whether 
this approach succeeds. 
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Chapter 4 

Reward Receipt in Family Rewards 
1.0 and 2.0 and a Look Ahead 


Introduction 

This report has described the evolution of the model of the conditional cash transfer program from 
Family Rewards 1 .0 to 2.0. Both programs offered cash incentives to parents and children in low- 
income families in the areas of children’s education, family preventive health, and adult employ- 
ment with the goals of relieving immediate hardship and attempting to break the cycle of inter- 
generational poverty by encouraging families to invest in their own development. However, the 
second version of Family Rewards shortened and revised the list of incentives, reserved eligibility 
for education rewards to high school students, targeted families receiving benefits from Tempo- 
rary Assistance for Needy Families (TANF) or the Supplemental Nutrition Assistance Program 
(food stamps), delivered the program in a new city, and added a guidance component. 

Because Family Rewards 2.0 is not a direct replication of the first program, the evalua- 
tion team treats comparisons between the families who participated in the two studies as 
suggestive but not conclusive evidence of how participants would have fared under different 
conditions. At the same time, each successive test of the conditional cash transfer concept 
affords new opportunities to track the performance of all program group members and those in 
particular subgroups, which may help refine the targeting and design of this kind of interven- 
tion. This chapter considers whether there are signs that the changes that were made to the 
model are increasing the receipt of rewards among all families, particularly among adults who 
did not have a high school diploma or General Educational Development (GED) certificate 
when they enrolled and among young people who entered high school scoring below proficien- 
cy in math or reading.' In the case of these subgroups, it was hoped that the program’s extra 
services and streamlined list of incentives would lead to higher receipt of education and work- 
force rewards, and eventually to larger effects on such families’ school and employment out- 
comes than were seen in Family Rewards 1 .0. 


*In 2010, the New York State Department of Education raised the scores necessary to be deemed profi- 
cient. As a result, proficiency rates will be much lower for the Family Rewards 2.0 sample than they were for 
the Family Rewards 1 .0 sample. 
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Key Findings 

• Families in Family Rewards 2.0 earned less money than families in the first 
program due to the shorter list of incentives and the exclusion of elementary 
and middle school students from eligibility for education rewards. This may 
have an impact on the program’s ability to reduce material hardship and on 
parent-child dynamics, since a large portion of the family’s annual reward 
earnings went to students for education rewards. 

• The program seems to be better equipped to engage families who are receiv- 
ing TANF and food stamps than Family Rewards 1.0. These families’ levels 
of reward receipt were higher in Family Rewards 2.0 for almost every type of 
reward that was offered by both programs. This is very likely the result of 
Advisors’ persistent outreach to low-earning families, although the improved 
economy may also have contributed to adult employment. The characteristics 
of the highest-earning families were the same in both programs and favored 
families who were larger and less disadvantaged at the start of the study. 


Comparing Family Rewards 1.0 and 2.0 Reward Receipt 

By some measures, the Family Rewards 2.0 sample should be harder to serve than the sample 
from Family Rewards 1.0 because all families had to be receiving TANF or food stamps at the 
time of enrollment. In Family Rewards 1.0, about 13 percent of the sample did not collect any 
public benefits or housing assistance, and in the first two years of that program these families 
made up 44 percent of the top tenth of the reward-earnings distribution.^ This was part of an 
overall trend where families who were relatively more advantaged earned more rewards. 

Overall Reward Earnings 

Table 4.1 compares the full Family Rewards 2.0 sample (Memphis and the Bronx) to 
only those families in Family Rewards 1 .0 who were receiving TANF and food stamps and who 
had a high school student at the start of the program (“the matching subsef ’). This subset of the 
Family Rewards 1 .0 sample is most similar to the current sample, and the table compares only 
Year 2 reward earnings in both programs.^ 


^Riccio (2010). 

^The trends described in this section are consistent when comparing the rewards earned by families from 
the Bronx in Family Rewards 1.0 and 2.0 who were receiving TANF and food stamps and who had high 
school students at the start of the program. See Appendix Tables D.l and D.2. 



SIF Family Rewards 
Table 4.1 


Comparison of Rewards Earned by Families 
in Year 2 of Family Rewards 1,0 and 2,0 


Outcome 

2.0 

1.0 

TANF/Food Stamp 
Participants”* 

Family earned at least 1 reward (%) 

96.3 

98.9 

Education reward 

93.1 

92.2 

Health reward 

78.8 

95.3 

Workforce reward 

43.7 

33.8 

Among families who earned at least 1 reward 

Average reward amount earned^ ($) 

2,160 

3,486 

Percentage of earnings from student rewards 
Average reward amount earned, by domain ($) 

57.1 

33.1 

Education 

1,146 

1,906 

Health 

661 

1,290 

Workforce 

1,129 

1,367 

Family picked up coupon book (%) 

95.0 

88.1 

Sample size 

1,230 

551 


SOURCES: MDRC calculations using CAS's Family Rewards 2.0 program data and 
Seedco's Family Rewards 1.0 program data. 

NOTES: Sample sizes may vary because of missing values. 

Year 2 in Family Rewards 2.0 covers reward activities that occurred between 
September 2012 and August 2013. Year 2 in Family Rewards 1.0 covers reward 
activities that occurred between September 2008 and August 2009. 

Italics indicate outcomes calculated for a subset of the full sample. 

“Includes families from Family Rewards 1.0 sample who were receiving TANF or 
food stamp benefits at the time of enrollment and had a high school student in the 
household in Year 2. 

'’The lowest amount earned in Year 2 in the 2.0 sample was $38 and the highest 
amount was $7,250; in the 1.0 TANF/food stamp sample the lowest amount earned 
was $80 and the highest amount earned was $12,525. 

‘’Percentage of earnings from student rewards is calculated using education rewards 
earned by high school students among families who earned rewards and had students 
who earned education rewards in Year 2. 


In the second program year, Family Rewards 2.0 families who earned rewards earned 
less money from the program than Family Rewards 1.0 families: $2,160 compared with $3,486. 
This was frue even though the percentage of families in Family Rewards 2.0 who earned at least 
one reward was about the same or higher in every domain other than health. Family Rewards 
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1.0 families who earned rewards earned an average of $760 more for education rewards, $629 
more for health rewards, and $238 more for workforce rewards. These large differences are 
mostly explained by the more generous high school student education rewards in the first 
program combined with the extra rewards families could earn for middle and elementary school 
children’s educational performance, and for maintaining public or private health insurance. The 
reward for maintaining public health insurance was worth a maximum of $100 (paid out every 
other month), and almost every family received the payment because almost all had health 
insurance coverage and because the reward was automatically verified (which eliminated the 
chance that participants might fail to claim it). As a result, almost the entire Family Rewards 1 .0 
matching subset earned at least one reward (about 99 percent), including at least one health 
reward (about 95 percent). 

Despite this high rate of earning health rewards, the Family Rewards 1.0 matching sub- 
set earned the largest share of their average annual earnings from education rewards. Again, 
Family Rewards 1.0 students earned much more than 2.0 students in that domain: $1,906 
compared with $1,146. Yet high school students’ education rewards made up 57 percent of 
average household earnings in Family Rewards 2.0 compared with 33 percent in 1.0. The key 
question for the impact study is whether the poverty-reduction effects of the first demonstration 
will continue even though the reward-receipt levels are much lower, and even though a much 
higher proportion of average family earnings is being earned by students and paid to students."^ 

Levels of Receipt of the Same Rewards 

While the Family Rewards 2.0 sample earned less money from the program, those par- 
ticipants were more likely to earn rewards that were similar to the ones offered in the original 
model, and the amounts earned for those categories of rewards were generally higher. Table 4.2 
provides another comparison of the Family Rewards 1.0 and 2.0 samples, in this case attempt- 
ing to illustrate how much families in the matching subset of Family Rewards 1 .0 would have 
earned if they only had access to the rewards available in Family Rewards 2.0.^ The difference 
in reward receipt favors the 2.0 sample in every domain, especially health. 

In Year 2, 96 percent of families in Family Rewards 2.0 earned at least one reward. 
Ninety-five percent of Family Rewards 2.0 families either went to the neighborhood partner 
organization (NPO) to get their coupon books or had them delivered by Advisors, compared 
with about 88 percent in Family Rewards 1.0 (in which the program staff distributed them at the 
NPO and by certified mail). This is an important indicator, since families who do not have a 

"^Many students’ accounts are jointly held with their parents because the students were minors at the start 
of the program. This raises an issue of potential conflict and confiscation of earnings by parents, which the 
implementation research will continue to follow. 

^Note that Table 4.2 includes the reward for grades, which was only available in Family Rewards 2.0. 


90 



SIF Family Rewards 
Table 4.2 


Comparison of Rewards Earned by Families in Year 2 of Family 
Rewards 1,0 and 2,0 for Rewards Offered by Both Programs 


Outcome 

2.0 

1.0 

TANF/Food Stamp 
Participants” 

Family earned at least 1 reward (%) 

96.3 

88.7 

Education reward 

93.1 

71.9 

Attendance 

88.6 

58.8 

State core exam 

62.8 

55.2 

College entrance exam’’ 

6.2 

9.8 

Report card 

80.2 

NA 

Health reward 

78.8 

64.4 

Annual physical 

70.4 

55.7 

Biannual dental 

70.7 

48.6 

Workforce reward 

43.7 

33.8 

Full-time employment 

43.3 

32.8 

GED certificate/employment training 

0.8 

2.2 

Among families who earned at least 1 reward 

Average reward amount earned‘^ ($) 

2,160 

1,640 

Percentage of earnings from student rewards 

57.1 

66.5 

Average reward amount earned, by domain ($) 

Education 

1,146 

1,127 

Health 

661 

286 

Workforce 

1,129 

1,367 

Family picked up coupon book (%) 

95.0 

88.1 

Family earned a reward in” (%) 

Year 1 but not Year 2 

1.8 

6.2 

Year 2 but not Year 1 

3.3 

3.6 

Year 1 and Year 2 

93.1 

84.6 


Sample size 1,230 551 

(continued) 
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Table 4,2 (continued) 


SOURCES: MDRC calculations using CAS's Family Rewards 2.0 program data 
and Seedco's Family Rewards 1.0 program data. 

NOTES: Sample sizes may vary because of missing values. 

Year 2 in Family Rewards 2.0 covers reward activities that occurred between 
September 2012 and August 2013. Year 2 in Family Rewards 1.0 covers reward 
activities that occurred between September 2008 and August 2009. 

Italics indicate outcomes calculated for a subset of the full sample. 

Measures for Family Rewards 1.0 are calculated using only rewards 
comparable to those in Family Rewards 2.0. These include rewards for state core 
exams, attendance, and PSAT exams for high school students; annual physical 
and biannual dental visits for all family members; and full-time employment and 
employment training rewards for adults. 

“Includes families from the Family Rewards 1.0 sample who were receiving 
TANF or food stamp benefits at the time of enrollment and had a high school 
student in the household in Year 2. 

'’College preparatory exams eligible for rewards included the PSAT for 
Family Rewards 1.0 and the SAT and ACT for Family Rewards 2.0. 

‘’The lowest amount earned in Year 2 in the 2.0 sample was $38 and the 
highest amount was $7,250; in the 1.0 TANF/food stamp sample the lowest 
amount earned was $100 and the highest amount earned was $6,050. 

^Percentage of earnings from student rewards is calculated using education 
rewards earned by high school students among families who earned rewards and 
had students who earned education rewards in Year 2. 

‘’Includes families from the Family Rewards 1.0 sample who were receiving 
TANF/food stamp benefits at the time of enrollment, and had a high school 
student in Year 1 and Year 2. 

coupon book cannot earn most of the rewards and are expressing a lack of interest in the offer. 
Table 4.2 also shows that fewer Family Rewards 2.0 families failed to earn any rewards in 
Years 1 and 2 of the program. 

Compared with the matching subset, a higher percentage of families in Family Rewards 
2.0 earned at least one reward for every activity except taking a college entrance exam and 
earning a GED certificate or completing employment training. Most notable is the difference in 
receipt of medical and dental rewards. About 70 percent of the Family Rewards 2.0 sample 
earned the medical and dental rewards at least once, compared with 56 and 49 percent, respec- 
tively, of the Family Rewards 1.0 matching subset. This shows what a substantial role the 
automatically verified health insurance reward played in boosting the percentage of Family 
Rewards 1 .0 families who earned a health reward and the amount of money families earned in 
the first program. The higher rate at which families earned medical and dental rewards in 
Family Rewards 2.0 was likely the result of a stronger focus on these rewards (given the shorter 
list of eligible activities), the large percentage of children in Memphis who claimed them, and 
Advisors’ persistent marketing efforts. 
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The earnings pattern with respect to full-time employment is noteworthy as well. Forty- 
four percent of Family Rewards 2.0 families earned the full-time work reward compared with 
34 percent of the adults in the Family Rewards 1 .0 matching subset. While the overall level of 
receipt of the full-time work reward by Family Rewards 2.0 parents is not as high as might be 
hoped, the fact that it exceeds the level achieved in the earlier program could reflect an improv- 
ing economy or could suggest that the new program model has succeeded in improving the 
marketing or accessibility of this reward. It should be noted, however, that the average amount 
of money earned from the workforce domain was slightly lower for Family Rewards 2.0 parents 
even though the dollar value of this reward did not change. The workforce domain includes the 
reward for employment training, so this small difference could be explained by more Family 
Rewards 1.0 parents claiming that reward, or it could indicate that Family Rewards 1.0 parents 
worked somewhat more regularly than 2.0 parents. The impact report will investigate whether 
parents in the Family Rewards 2.0 sample have stronger workforce impacts (for example, more 
employment, wages, etc.) than were seen in Family Rewards 1.0. 


Patterns of Earning Based on Participant Characteristics 

The Family Rewards 1.0 implementation study found that the families who earned the most 
rewards were, in general, larger and less disadvantaged than the sample as a whole. The heads 
of household in these families had, on average, higher levels of education, employment, and 
earnings from employment; higher marriage rates; and better self-reported mental and physical 
health than the heads of other families in the program group sample. These families had more 
stable housing and — especially relevant to this comparison — were less likely to be receiving 
public benefits. Language and ethnicity did not affect earnings, but most foreign-bom families 
were in the top half of earners. That study did not examine the characteristics of children in 
higher-earning families. 

All of these trends were repeated in the current study. '’Larger families earned more re- 
wards. In Family Rewards 2.0, the average annual amount earned in Years 1 and 2 by families 
with one child was $2,781 compared with $4,370 for families with four or more children (data 
not shown in a table). Table 4.3 shows the distribution of household earnings by certain baseline 
characteristics, and shows that those in the top 20 percent of reward earnings ($5,443 to 
$14,151) were more likely to be married and had about one more child in the household than 


'’To more closely match the analyses performed for Family Rewards 1.0, many of the following tables 
combine Year 1 and 2 reward earnings from Family Rewards 2.0. 
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SIF Family Rewards 
Table 4,3 


Baseline Characteristics of Families, 
by Range of Reward Earnings During Program Years 1 and 2 


Characteristic 

Bottom 20% 
$0- $1,560 

Middle 60% 
$1,561 -$5,442 

Top 20% 
$5,443 -$14,151 

Baseline characteristics of adults^ 




No high school diploma or GED certificate (%) 

47.1 

39.5 

36.0 

Foreign-bom (%) 

21.5 

25.4 

44.3 

Currently employed (%) 

30.1 

48.6 

76.3 

Full time (at least 30 hours per week) 

17.1 

33.8 

61.2 

Part time 

12.7 

14.4 

15.1 

Has a physical problem that limits work (%) 

26.0 

16.5 

5.7 

Has an emotional or mental health problem that limits work (%) 

10.3 

6.7 

1.2 

At risk of depression (%) 

31.4 

22.9 

13.6 

Baseline characteristics of families 




Average number of children in household 

2.4 

2.7 

3.3 

Housing status (%) 




Lives in rental housing 

79.6 

81.6 

86.9 

Lives in public housing 

22.1 

21.7 

23.1 

Married, living with spouse (%) 

13.4 

12.2 

21.1 


(continued) 



Table 4,3 (continued) 


Characteristic 

Bottom 20% 
$0- $1,560 

Middle 60% 
$1,561 -$5,442 

Top 20% 
$5,443 -$14,151 

Receiving benefits (%) 

TANF 

21.1 

18.6 

13.9 

Food stamps 

99.2 

98.4 

98.0 

Program characteristics of adults“ 

Participant received at least 1 interaction (%) 

93.9 

99.3 

99.6 

Guidance'’ 

55.1 

85.9 

91.1 

Customer service'’ 

47.4 

78.7 

78.9 

Outreach'' 

91.9 

93.4 

91.1 

Among those who received at least 1 of eaeh specified category 

Average number of interactions received 

5.8 

6.9 

6.5 

Guidance 

1.7 

1.8 

1.7 

Customer service " 

1.8 

2.6 

3.0 

Outreach 

4.7 

4.8 

4.3 

Advisor had high profieiency in motivational interviewing'’ (%) 

51.1 

47.3 

49.0 

Sample size (total = 1,230) 

247 

737 

246 
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Table 4,3 (continued) 


SOURCES: MDRC calculations using data from CAS's Family Rewards program data and baseline survey data. 

NOTES: Sample sizes may vary because of missing values. 

Year 1 covers reward activities from September 2011 through August 2012. Year 2 uses data from September 2012 
through August 2013. 

Italics indicate outcomes calculated for a subset of the full sample. 

“For two-parent households, the eharacteristics of one adult were selected. For households with two adults, the female 
adult was selected as the representative adult of the household because the majority of single-parent households were 
headed by female adults. For cases where both adults were female, the representative adult was selected at random. 

'’Guidance includes: creating Family Earning Plan, reviewing Family Earning Plan, discussing next steps, discussing 
barriers to earning, discussing public benefits, and making a referral. 

‘’Customer service includes: group check-in, group workshop, discussing referral outcome, submitting bank account 
information, adjusting family composition/eligibility. Advisor signature for GED certificate completion, answering 
earnings questions, help-line call, and coupon submission support. 

‘'Outreach includes: program eampaign (outreach to particular participants with information or reminders), scheduling 
coupon book pick-up, program reengagement call, sending e-mail, sending letter, and conducting automated call. 

“Only one Advisor was assigned per family. Advisor assignment was only available for families who received contact 
from an Advisor. 



the lowest 20 percent of reward earners. These families were also more likely to be led by adults 
who had at least a high school diploma or GED certificate, who were working full time at the 
time they enrolled in the study, and who reported being in good health, both mentally and 
physically. Foreign-bom adults headed 44 percent of the highest-earning families, while 
families receiving TANF were somewhat more likely to be among the middle and low earners. 

For students (Table 4.4), the data show that female students with higher academic pro- 
ficiency and better health were more likely to be among the highest earners for education 
rewards. While these trends demonstrate the significant role that preexisting characteristics play 
in determining how much participants earn from a financial incentives program, they should not 
overshadow the ways the current program raised participation levels in general for this relative- 
ly hard-to-serve population, described in the previous section. 


SIF Family Rewards 
Table 4.4 


Baseline Characteristics of High School Students, 
by Range of Reward Earnings During Program Years 1 and 2 


Characteristic (%) 

Bottom 20% 
so - $400 

Middle 60% 
$401 -$2,140 

Top 20% 
$2,141 -$4,250 

Student proficiency levef 

Nonproficient in English 

93.7 

87.3 

58.7 

Nonproficient in math 

94.8 

86.1 

41.3 

Female 

41.1 

50.6 

62.4 

General health status 

Excellent/very good 

66.5 

76.4 

77.6 

Good 

28.6 

21.2 

20.6 

Fair/poor 

4.8 

2.4 

1.6 

Individualized Education Program 

24.4 

15.4 

7.6 

Sample size (total = 1,455) 

270 

863 

322 


SOURCES; MDRC calculations using CAS's Family Rewards program data and baseline survey data. 

NOTES; Sample sizes may vary because of missing values. 

Year 1 covers reward activities from September 201 1 through August 2012. Year 2 covers reward 
activities from September 2012 through August 2013. 

“Proficiency level is only reported for high school students who had taken a standardized test to 
determine proficiency within the two years prior to enrollment. Data were available for most students who 
were in ninth or tenth grade at the time of enrollment. 
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The designers of Family Rewards 2.0 were particularly interested in improving partici- 
pation and program impacts for two subgroups at risk of not earning education and workforce 
rewards: students who were not academically proficient when they entered the study and adults 
who did not have a high school diploma or GED certificate when they entered the study. Table 
4.5 shows the percentage of each of these subgroups who earned rewards for attendance and 
passing their state exams (among students), and for working full time and training or obtaining a 
GED certificate (among adults) in Year 2 of both programs. Across the board, higher percent- 
ages of academically proficient and nonproficient students earned these rewards in Family 
Rewards 2.0 than in 1 .0. For example, among students who were nonproficient in English at the 
start of the program, 87 percent earned at least one attendance reward and 54 percent earned at 
least one reward for passing a state exam in Family Rewards 2.0. In contrast, in Family Re- 
wards 1.0 only 55 percent of this subgroup of students in the matching subset earned an attend- 
ance reward, and 39 percent earned a state exam reward.’ Among parents without a high school 
diploma or GED certificate, 38 percent earned at least one reward for full-time work in Family 
Rewards 2.0 compared with 27 percent in Family Rewards 1.0. 

An important factor in this increased participation seems to be the Advisors’ persistent 
outreach efforts.* The bottom panel of Table 4.3 shows the percentage of adults who were 
reached by their Advisors and how many interactions, on average, adults had with their Advi- 
sors."^ Families who earned the most rewards were those with adults who had the most contact 
with Advisors. Ninety-one percent of the adults in the highest-earning group experienced a 
guidance session, and among adults who received at least one guidance session, the average 
number of guidance sessions was about two. The corresponding figure among adults in the 
lowest-earning families is 55 percent. The amount of customer service adults received also 
increases with family earnings, which is logical since there is a direct connection between 
receiving customer service and claiming rewards. The most interesting finding with respect to 
staff interactions is that adults in families who had the lowest reward earnings received about as 
much outreach as adults in families in the middle and at the top of the reward-earning distribu- 
tion. Ninety-two percent of adults in the lowest-earning families received some form of 


’in 2010, the New York State Department of Education raised the scores necessary to be deemed “profi- 
cient.” As a result, proficiency rates will be much lower for the Family Rewards 2.0 Bronx sample than they 
were in the Family Rewards 1.0 sample. The findings reported here were similar when proficiency was 
determined for the Family Rewards 1.0 sample using the new, higher score requirements. 

*There is no apparent correlation between Advisor competency in motivational interviewing and family 
placement on the reward earnings distribution. This finding should be treated with caution because a simple 
correlation like the one presented in Table 4.3 may not be sensitive enough to uncover the influence of this 
approach on reward receipt. The relationship between Advisor quality and participant outcomes will be 
investigated further in the next report. 

’For two-parent households, the characteristics of one adult were selected. The notes to the table explain 
how that adult was chosen. 
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SIF Family Rewards 
Table 4.5 


Comparison of Reward Earnings in Year 2 
of Family Rewards 1,0 and 2,0, by Subgroup 



2.0 

1.0 

Reward Earned (%) 


TANF/Food Stamp 
Participants” 

Among English-nonproficient students'’ 

Attendance 

86.8 

54.6 

State core exam 

53.8 

39.2 

Sample size 

982 

335 

Among English-proficient students*’ 

Attendance 

93.2 

72.9 

State core exam 

81.1 

79.1 

Sample size 

222 

129 

Among adults without GED certificates or high school diplomas*’ 

GED certificate/employment training 

1.6 

0.6 

Full-time employment 

37.7 

26.2 

Sample size 

510 

507 

Among adults with GED certificates or high school diplomas*’ 

GED certificate/employment training** 

0.3 

2.2 

Full-time employment 

45.4 

39.6 

Sample size 

760 

371 


SOURCES: MDRC calculations using Seedco's Family Rewards 1.0 program data and CAS's Family 
Rewards 2.0 program data. 

NOTES: Sample sizes may vary because of missing values. 

Year 2 in Family Rewards 2.0 covers reward activities that occurred between September 2012 and 
August 2013. Year 2 in Family Rewards 1.0 covers reward activities that occurred between 
September 2008 and August 2009. 

^Includes participants from families who were receiving TANF or food stamp benefits at the start 
of the study. 

’’Proficiency level is only reported for high school students who had taken a standardized test to 
determine proficiency within the two years prior to enrollment. Data were available for most students 
who were in eighth or ninth grade when they enrolled in Family Rewards 1.0 and for most students 
who were in ninth or tenth grade when they enrolled in Family Rewards 2.0. 

‘’Excludes adults who did not provide their education levels at the time of enrollment. 

‘’Participants in Family Rewards 2.0 with a GED certificate, high school diploma, or higher 
should not have been eligible to earn a GED reward. 
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outreach in Year 2 of the program, and among adults who received outreach, the average 
number of contacts was about five. This is very similar to the 91 percent of the highest-earning 
group who received outreach, averaging four contacts. It is very different, however, from 
Family Rewards 1.0. In that program, the staff was trained to provide customer service and 
referrals reactively, only when participants initiated contact. Here the Advisors pursued low- 
earning families through multiple channels including phone calls and letters, and conducted in- 
person meetings in their homes or at the NPO when they could. 

Until the impact evaluation is complete, it is not possible to know whether the current 
program is changing participants’ outcomes in the areas targeted by the intervention. However, 
it seems clear from the foregoing that the current program design has improved levels of reward 
receipt in all areas, especially health, and has increased the proportion of the sample who are 
showing interest in the program by picking up coupon books, earning rewards each year, and 
earning different kinds of rewards. While the level of participation in the workforce reward is 
still not as high as the levels of reward receipt in the other domains, it too has improved com- 
pared with Family Rewards 1 .0 for adults receiving public benefits. All of these are encouraging 
signs of the potential effectiveness of this program model. 


Looking Ahead 

This report covered the first two years of Family Rewards 2.0, which to date has been a story of 
continuous program improvement. The Children’s Aid Society (CAS)-Central, the Family 
Rewards Collaborative, and the NPOs have guided the program through a process of monitor- 
ing its own performance and making adjustments to improve the simplicity and accessibility of 
the rewards and the intensity of the family guidance. The program operators have engaged in 
campaigns to market the rewards that were responsive to participants’ experiences, especially 
the experience of having coupons rejected by the payment-processing system. The management 
information system has improved from year to year to allow the program to more precisely 
target participants who did not earn rewards. 

Because these changes coalesced in the second part of Year 2, the final year of the pro- 
gram will be critical for evaluating the success of these initiatives. While the program was 
originally expected to offer incentives from September 2011 to August 2014 (36 months), the 
delay in recmiting the sample prompted the Family Rewards Collaborative to shift the timeta- 
ble: the program has been extended through December 2014, an addition of 4 months. All 
rewards will be available during the 4-month extension to all family members who were 
enrolled in Year 3, and self-employed adults will have the opportunity to submit their 2014 tax 
returns to receive the workforce reward in early 2015. 
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The next report on this project will be a combined impact and implementation report 
that covers the program until the originally expected end date of August 2014, at minimum, and 
that includes administrative and survey data to describe program impacts in the areas of family 
well-being, education, health, and employment. In addition, the implementation study will 
conduct observations of program activities and a final round of interviews with program 
participants (adults and high school students). Advisors, and staff members from CAS-Central. 
These qualitative data sources will be combined with survey data to provide a more thorough 
analysis of changes, if any, in participants’ behavior, motivation, and family dynamics due to 
the program. The study will continue to focus on the contribution of family guidance to these 
changes, with a stronger emphasis on the connection between participant outcomes and guid- 
ance experiences. Depending on funding, additional reports may follow. 
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Supplementary Exhibits for Chapter 1 




SIF Family Rewards 


Appendix Table A.l 

Comparison of Rewards Offered by Family Rewards 1.0 and 2.0 


Reward Type 

Family Rewards 1.0 

Family Rewards 2.0 



Bronx 

Memphis 

Education incentives 




Elementary and middle school students (paid to parent) 




Attends 95% of scheduled school days every month 

$25 per month (discontinued after Year 2) 

Not offered 

Not offered 

Scores at proficiency level or improves 
on annual math and English tests 




Elementary school students 

$300 per math test; $300 per English test 

Not offered 

Not offered 

Middle school students 

$350 per math test; $350 per English test 

Not offered 

Not offered 

Parent reviews low-stakes interim tests 
up to 5 times per year 

$25 for parents to download, print, and 
review results (discontinued after Year 1) 

Not offered 

Not offered 

Parent discusses up to 2 annual math and 
English test results with teachers 

$25 (discontinued after Year 2) 

Not offered 

Not offered 

High school students^ 




Takes up to 2 PSAT tests 

$50 per test 

Not offered 

Not offered 

Accumulates 1 1 course credits per year 

$600 

Not offered 

Not offered 

Graduates from high school 

$400 

Not offered 

Not offered 

Attends 95% of scheduled school days 

$50 per month 

$40 per month 

$40 per month 

Takes an SAT or ACT exam (once during program) 

Not offered 

$50 

$50; must score 19 or 
better on the ACT if 
administered by MCS*” 


(continued) 



Appendix Table A.l (continued) 


Reward Type 

Family Rewards 1.0 

Family Rewards 2.0 



Bronx 

Memphis 

Receives grades on an official report card 

Not offered 

$30 per A (90-100); 
$20 per B (80-89); 
$10 per C (75-79) 

$30 per A (93-100); 
$20 per B (85-92); 
$10 per C (75-84)" 

Passes up to 5 Regents exams'^ or 7 End of 

$600 per Regents exam 

$500 per Regents 

$200 per End of 

Course exams'^ 


exam for a score of 
75 or above; $400 
per exam for a 
score of 65-74 

Course exam for a 
score of proficient or 
advanced (increased 
to $300 in Year 2) 

All grades 

Parent attends parent-teacher conferences 
up to 2 times per year 

$25 per conference 

Not offered 

Not offered 

Child obtains library card 

$50 once during program 
{discontinued after Year 2) 

Not offered 

Not offered 

Health incentives 

Maintaining public or private health insurance 
For each parent covered 
If all children are covered 

Per month: $20 (public); $50 (private) 

Per month: $20 (public); $50 (private) 
{discontinued after Year 2) 

Not offered 

Not offered 

Annual medical checkup 

$200 per family member 

$100 per family 
member 

$100 per family 
member 

A doctor-recommended follow-up visit 

$100 per family member 
(discontinued after Year 2) 

Not offered 

Not offered 

Early-intervention evaluation for child under 30 
months old, if advised by pediatrician 

$200 per child 

Not offered 

Not offered 
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Appendix Table A,1 (continued) 


Reward Type 

Family Rewards 1.0 

Family Rewards 2.0 



Bronx 

Memphis 

Preventive dental care every 6 months 
(once per year for children 1 -5 years old) 

$100 per family member 

$100 per family 
member per visit 

$100 per family member 
per visit 

Adult workforce efforts 

Sustained full-time employment 

$150 per month 

$150 per month 

$150 per month 

Education and training while employed at least 
10 hours per week 

Amount varied by length of course, 
up to a maximum of $3,000 over 3 years 
{employment requirement discontinued 
after Year 2) 

Not offered 

Not offered 

GED certificate 

Not offered 

$400 for successful 
completion 

$400 for successful 
completion 


SOURCE: Seedco's Family Rewards 1.0 program materials and Family Rewards 2.0 program materials from the Children's Aid Society (CAS). 

NOTES: GED = General Educational Development; MCS = Memphis City Schools. 

“In Family Rewards 1.0, the following education rewards were paid entirely to students: taking up to two PSAT Tests ($50), passing five Regents 
exams ($600 per exam), and obtaining a library card ($50). The following education rewards were split between parents and students: accumulating 1 1 
course credits per year ($600) and attending 95 percent of scheduled school days. In Family Rewards 2.0, all education rewards were paid entirely to 
students. 

'’The ACT is scored out of 36. Memphis City Schools officials requested that the minimum score for the reward be set at 19 for students taking 
the test for free in class because this is considered an indication that students are ready for college-level work. 

Tn Year 1, the amount of each reward for grades was prorated based on the number of official report cards issued by a student's school. To 
simplify verification in Year 2, students were paid the listed amounts for their grades regardless of the number of official report cards they 
received, up to a maximum of $600 per program year. 

‘'High school students (grades 9-12) in the Bronx were eligible to earn rewards for the following Regents exams: English, one of any math exams 
(including Math A, Math B, Integrated Algebra, Geometry, and Algebra 2/Trigonometry), U.S. History and Government, Global History and Geography, 
and one of any science exams (including Living Environment, Chemistry, Physics, and Earth Science). 

“High school students (grades 9-12) in Memphis were eligible to earn rewards for the following state core exams: Algebra 1, Algebra 2, Biology, 
English 1, English 2, English 3, and U.S. History. 
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SIF Family Rewards 
Appendix Table B.l 


Year 2 Group Events and Campaigns in Memphis and the Bronx 


Quarter 1 2013 

Neighborhood 

Partner 

Organization 

Target 

Group 

Event 

Porter-Leath 

Participants 
who had not 
submitted 
Year 2 dental 
coupon 

Eggstravaganza 

Firestone, a dental partner, was on-site to complete eleanings. Step Ahead 
Foundation discussed health and wellness with interested teens. 

Hosted by 
Porter-Leath 
but Memphis 
HOPE 
participants 
could attend 

All adult 
participants 
with part-time 
or 

unemployed 

status 

Resume Writing Workshop 
Local organizations were present. 

Memphis 

HOPE 

All students 
with high or 
low earnings 

2 Day Spring Bling 

Summer employment, state core exams, college prep, and summer 
enrichment activities were discussed. Memphis HOPE also gathered 
information on the best ways to engage teens for increased participation. 

Quarter 2 2013 

Hosted by 
Memphis 
HOPE but 
Porter-Leath 
participants 
could attend 

Students and 
parents 

Summer BBQ 

Advisors set up a table to help participants with coupon submission, 
conducted screenings and accepted applications for Porter-Leath's early 
education program. 

Children's Aid 
Society (CAS)- 
NPO 

Students and 
Parents 

Mother's Day and Wellness Celebration 
A speaker from Fordham School of Law's Coalition for Debtor Education 
discussed financial debt. Staff members from CAS-Central's Parent 
Leadership Institute made a presentation on their organization's services. 
Representatives from Affinity Healthcare distributed information. 

Hosted by 
CAS-NPO but 
BronxWorks 
participants 
could attend 

Students 

Financial Literacy Workshop-Facilitated by Hope Leadership's Peer 
Educator Students 

Students learned about saving, credit card debt, taxes, and needs versus 
wants. The group facilitators reviewed two budgeting scenarios in whieh 
participants discussed how they thought the characters could change their 
behaviors in order to save money. Students also discussed what "wealth" 
meant to them. 

BronxWorks 

Students 

Knowledge for College Event 

The event featured seven current college students who had graduated from 
BronxWorks CAFE (Center for Achieving Future Education) program. 
Students worked one-on-one with families to discuss affordability, 
scholarships, timetables for financial aid, loans, obtaining college credit in 
high school, and the importance of parental involvement. 


(continued) 
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Appendix Table B,1 (continued) 


BronxWorks 

Students 

College Fair at the Jacob Javitz Center 
Staff members worked with each student to identify two potential majors 
so students could make informed decisions about potential colleges. 
Students attended a follow-up college prep group on May 4. 

Quarter 3 2013 

BronxWorks 

All families 

Summer Jam: Back to School Event 
Featured music, food, a face painter, a dance contest, and spoken-word 
performance. School supplies, book bags, and health and wellness 
products were available to families. Advisors distributed Y ear 3 coupon 
books to families who had not attended their previously scheduled 
appointments. 

Quarter 4 2013 

Porter-Leath 

Students 

How to Get Financial Aid for College 
CAS-Central partnered with the Tennessee Student Assistance 
Corporation (TSAC), an organization that provides information on 
financing college in the state of Tennessee. TSAC conducted an 
informational session for that provided an overview of available 
scholarships and need-based funding. Families also received hands-on 
assistance completing the Free Application for Federal Student Aid 
(FAFSA). 

All Year 

Campaigns: rejected work coupons, students earning less than $75, adults who earned in Year 1 and not Year 
2, students who earned in Year 1 and not Year 2, rejected health coupons, and distributing coupon books. 


SOURCE: CAS's Family Rewards program materials. 
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SIF Family Rewards 
Appendix Table B,2 

Education Rewards Earned by Students in Year 2, by Subgroup 


Outcome 

Bronx 

Memphis 

Students nonuroficient in English^ 



Earned any education reward (%) 

88.8 

95.3 

Among students who earned at least 1 reward 



Average total amount earned^ ($) 

1,031 

697 

State core exam passed (%) 

64.6 

45.2 

Attendance reward earned (%) 

79.6 

92.1 

7 or more attendance rewards earned 

36.5 

49.6 

Report card reward earned (%) 

75.2 

81.1 

Student earned at least 1 reward, by letter grade (%) 



A 

60.2 

66.0 

B 

70.7 

77.1 

C 

65.8 

79.1 

Sample size 

427 

555 

Students nroficient in English'* 



Earned any education reward (%) 

95.1 

100.0 

Among students who earned at least 1 reward 



Average total amount earned’^ ($) 

1,312 

1,117 

State core exam passed (%) 

78.9 

85.0 

Attendance reward earned (%) 

89.4 

100.0 

7 or more attendance rewards earned 

47.2 

68.8 

Report card reward earned (%) 

87.3 

91.3 

Students earned at least 1 reward, by letter grade (%) 



A 

79.6 

86.3 

B 

82.4 

90.0 

C 

67.6 

83.8 

Sample size 

142 

80 


(continued) 
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Appendix Table B,2 (continued) 


SOURCE: MDRC calculations using CAS's Family Rewards program data. 

NOTES: Sample sizes may vary because of missing values. 

Year 2 covers reward activities that occurred between September 2012 and August 2013. 

Italics indicate outcomes calculated for a subset of the full sample. 

“Proficiency level is only reported for high school students who had taken a standardized 
test to determine proficiency within the two years prior to enrollment. Data were available for 
most students who were in ninth or tenth grade at the time of enrollment and had taken a 
standardized test to determine proficiency within the prior two years. 

'’Students nonproficient in English: the lowest and highest amounts earned in Year 2 were 
$10 and $2,500 for Bronx participants, and $10 and $1,820 for Memphis participants, 
respectively. 

‘’Students proficient in English: the lowest and highest amounts earned in Year 2 were $40 
and $2,640 for Bronx participants, and $40 and $1,900 for Memphis participants, 
respectively. 
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SIF Family Rewards 
Appendix Table B,3 

Workforce Rewards Earned by Parents in Year 2, by Subgroup 


Outcome 

Bronx 

Memphis 

Parents without a degree at the start of the study 

Earned any workforce reward (%) 

40.7 

35.6 

Among parents who earned at least 1 workforce reward 
Average total amount earned ($) 

1,035 

995 

Full-time employment reward earned (%) 

40.3 

33.7 

Among parents who earned at least 1 employment reward 
Average number of months parent earned reward 

6.9 

6.8 

General Educational Development (GED) certificate reward earned"* (%) 

1.0 

2.4 

Sample size 

305 

205 

Parents with a degree at the start of the study 

Earned any workforce reward (%) 

46.9 

44.2 

Among parents who earned at least 1 workforce reward 
Average total amount earned ($) 

1,186 

1,155 

Full-time employment reward earned (%) 

46.9 

44.2 

Among parents who earned at least 1 employment reward 
Average number of months parent earned reward 

7.9 

7.7 

GED certificate reward earned"* (%) 

0.0 

0.5 

Sample size 

341 

419 


SOURCE: MDRC calculations using CAS's Family Rewards program data. 

NOTES: Sample sizes may vary because of missing values. 

Italics indicate outcomes calculated for a subset of the full sample. 

Year 2 covers reward activities that occurred between September 2012 and August 2013. 
Table only includes participants who reported their education levels on the baseline survey. 
‘‘Participants with a GED credential, high school diploma, or higher should not have been 
eligible to earn a GED reward. 
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SIF Family Rewards 
Appendix Figure B,1 
Theory of Change 


Rewarded Aetivities 


Mediating Faetors 



Middle- 

Term 

Outeome 


Long-Term Onteomes 


1 1 

1 1 

1 Increase in adult employment | 



Sustained 

' • Greater access to employment resources ' — ► 


— ► 

full-time 

1 • Fewer missed days of work i 

1 1 



employment 


Improved preventive health eare 

• Greater access to health and dental services 

• Fewer missed days of school or work 


Improved edneation onteomes 

• Higher attendance rate 

• Better academic performance 

• Increase motivation to achieve 

• Greater access to academic support 

• Change in time use (Morris, Aber, Wolf, 
and Berg, 2012) 


Rednetion of 
immediate 
eeonomie 
hardship 


Sustained 

good 

health-related 

habits 



Rednetion in 
2nd- 

generation 

poverty 



• Guidance 

• Family Resource Fund 

• Customer service 
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SIF Family Rewards 
Appendix Table C.l 


Family Earning Plans Created by tbe End of Year 2, 
by Neighborhood Partner Organization (NPO) 


Outcome 

Bronx Works 

Children's Aid 
Society 

(CAS)-NPO Memphis HOPE 

Porter-Leath 

Adults 





Created Earning Plan (%) 

94.4 

94.6 

89.5 

96.1 

Sample size 

342 

313 

323 

308 

Students'* 





Created Earning Plan (%) 

91.0 

89.6 

68.7 

86.4 

Sample size 

389 

365 

450 

419 


SOURCE: MDRC calculations using Family Rewards program data from the Children's Aid 
Society (CAS). 

NOTES: Year 2 covers activities that occurred between September 2012 and August 2013. 

“Sample only includes students who were identified as being enrolled in high school during Year 

2 . 
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SIF Family Rewards 
Appendix Figure C,1 
Sample Family Earning Plans 


K) 

O 


FAMILY 

REWARDS 


Adult Earnings Plan 


Name: 

Date: 


Individual ID: 

Family RewardsAdvisor: 



Another adult enrolled? Fill out a second Work Rewards Plan and maximize your earningsl 
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Appendix Figure C,1 (continued) 


Student Earnings Plan 

FAMILY 

REWARDS student Name; Individual ID; 

Date: Family RewardsAdvisor: 



Activity 

How rvluch 

Maximum 

Rewards 

when Do 1 Feel Ready to 
Earn? 

Student checks option 

Target Date and Next Steps 

To complete with your Advisor- 
Advisor: Attach Referral Tracker for next steos 

c 

0 

School 

Attendance 

No more than 1 
absence 

$40 

per month 

$400 

per year 

This month 

Next month 

Within six months 


Grades 

Have acopy of your 
re port card signed by 
your Family Rewards 
Advisor 

$30 per 93-100 
$20 per 85-92 
$10 per 75-84 

$600 

per year 

This report card 

M y next report card 

Next school year or later 


(Q 

0 

3 

■o 

u 

End of Course 
Exams 

Stude nt achieve s profi- 
cient or advanced level. 
Ten exams over 4 years of 
high school. 

$200 

$ 2,000 

in the 
Program 

Spring 2012 

Next school year or later 



SAT or ACT 



Junior year 



Stude nt score s 19 or > on 

$50 

$50 




MCS administered ACT 



Senior year 



exam; or studenttakes 

□nee in program 

in the 
program 




ACT or SAT independently 



I'm not sure 



A student can earn about^^^^Qfor completing activities in a full school year. 
Don't miss out! Start to earn this month ! 


SOURCE: CAS's Family Rewards program materials. 

NOTE: The Adult Earnings Plan above includes only the first page of the plan, presenting work incentives; the second page (not shown) lists 
health incentives. 
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SIF Family Rewards 
Appendix Figure C,2 

Year 2 Family Earning Plan Thermometer Sheet 


FAMILY 

REWARDS 


Name/ Date: 
individuaVFam^y ID: 


$$$ My Family Has Earned 
As of 




Potential 

Earnings 




$$$ EARNED 

$$$ LOST 

Family Plan to Earn 
More Rewards 

^ Work 

Full-Time Work 

$ 

S 


6ED 

S 

s 


1 Health 

Annual Physicals 

$ 

s 


Dental Check-Ups#! I 

For appointments brfore 2/28 1 

s 

s 


Dental Check-Ups #2 

For appointmentsfrom 2/28 to 8/31 

$ 

s 


Education 

Monthly Attendance | 

$ 

s 


Report Card Grades | 

$ 

s 


Regents Exams | 

$ 

s 


SAT/ACT 

$ 

s 


Total 

$ 

EARNED 

s 

LOST 



SOURCE: CAS's Family Rewards program materials. 
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SIF Family Rewards 
Appendix Figure C,3 
Sample Work-Readiness Tool 


When an adult indicates that he or she is not ready to earn Work Rewards this month, the 

following questions may help you to make an educated referral. 

• Have you worked any hours this month? Can you tell me about your work history? 

• If participant is working part- 

• Discuss the participant’s 

time, discuss any opportunities 

network 

to increase hours at their current 


job. 


Explore Barriers to Work 


A. Health (Mental and Physical) 


In many families we work with, parents have health 

concerns that affect their opportunity to 

work. Have you had any of these experiences recently? 

• Challenges learning and absorbing 

• A family crisis or very emotional 

new job skills 

event 

• Physical pain or injury 

• Violence or unsafe conditions in the 

• A dependence on drugs or alcohol 

home 

B, Language 


What languages do you speak? 


• Do you feel that your language skills have interfered with finding or keeping work? 

C. Housing Stability 


Moving a lot makes it difficult for some parents to get to work every day. I’m going to ask about 

your housing history. 


• How many times have you moved in 

• Do you expect to stay through the 

the last two years? 

year? 

• Did you experience any period of 

• Can you easily access public 

homelessness during this time? 

transportation from your home? 

• How long have you lived at your 


current address? 


D, Child Care 


Would you describe any help you currently have with child-care? 

• If you worked full-time, would your 

• Do any of your children have 

current help be available? 

challenges that make it difficult for 

• Do you have a back-up plan when 

them to stay in child-care? 

your primary help is unavailable? 



SOURCE: CAS's Family Rewards program materials. 
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SIF Family Rewards 
Appendix Figure C,4 
Sample Goal-Setting Tool 



FAMILY 


REWARDS 


GOAL SETTING 


Participant 




/ ) 


I 


V 


1 





SOURCE: CAS's Family Rewards program materials. 
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Percentage completing a review 


SIF Family Rewards 
Appendix Figure C,5 


Summary of Student Earning Plan Reviews in Year 2, 
by Neighborhood Partner Organization 


100 -| 
90 - 



BronxWorks CAS-NPO Memphis HOPE Porter-Leath 


■ Completed 1 review □ Completed 2 reviews 0 Completed more than 2 reviews 

SOURCE: MDRC calculations using CAS's Family Rewards program data. 

NOTE: Year 2 covers activities that occurred between September 2012 and August 2013. 
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Appendix D 

Supplementary Exhibits for Chapter 4 




SIF Family Rewards 
Appendix Table D.l 


Comparison of Rewards Earned by Bronx Families 
in Year 2 of Family Rewards 1.0 and 2.0 


Outcome 

2.0 

1.0 

TANF/Food Stamp 
Participants'* 

Family earned at least 1 reward (%) 

94.8 

98.9 

Education reward 

89.8 

91.6 

Health reward 

79.1 

95.8 

Workforce reward 

45.5 

37.4 

Among families who earned at least 1 reward 

Average reward amount earned^ ($) 

2,281 

3,626 

Percentage of earnings from student rewards 
Average reward amount earned, by domain ($) 

58.1 

36.0 

Education 

1,287 

2,078 

Health 

617 

1,237 

Workforce 

1,140 

1,337 

Family picked up coupon book (%) 

95.5 

89.5 

Sample size 

617 

190 


SOURCES: MDRC calculations using Seedco's Family Rewards 1.0 program data 
and Family Rewards 2.0 program data from the Children's Aid Society (CAS). 

NOTES: Sample sizes may vary because of missing values. 

Year 2 in Family Rewards 2.0 covers reward activities that occurred between 
September 2012 and August 2013. Year 2 in Family Rewards 1.0 covers reward 
activities that occurred between September 2008 and August 2009. 

Italics indicate outcomes calculated for a subset of the full sample. 

''Includes families from Family Rewards 1.0 sample who were receiving 
Temporary Assistance for Needy Families (TANF) or Supplemental Nutrition 
Assistance Program (food stamp) benefits and lived in the Bronx at the time of 
enrollment, and who had a high school student in the household in Year 2. 

'’The lowest amount earned in Year 2 in the Family Rewards 2.0 sample was $38 
and the highest amount was $7,250; in the 1.0 TANF/food stamp sample the lowest 
amount earned was $80 and the highest amount earned was $12,525. 

‘’Percentage of earnings from student rewards is calculated using education 
rewards earned by high school students among families who earned rewards and had 
students who earned education rewards in Year 2. 
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SIF Family Rewards 
Appendix Table D,2 


Comparison of Rewards Earned by Bronx Families in Year 2 
of Family Rewards 1.0 and 2,0 for Rewards Offered 
by Both Programs 


Outcome 

2.0 

1.0 

TANF/Food stamp 
Participants^ 

Family earned at least 1 reward (%) 

94.8 

88.9 

Education reward 

89.8 

74.2 

Attendance 

82.7 

63.7 

State core exam 

68.9 

61.1 

College entrance exam'’ 

9.1 

10.0 

Report card 

78.0 

NA 

Health reward 

79.1 

65.8 

Annual physical 

72.3 

54.2 

Biannual dental 

70.0 

52.1 

Workforce reward 

45.5 

37.4 

Full-time employment 

45.4 

36.8 

GED certificate/employment training 

0.5 

2.1 

Among families who earned at least 1 reward 

Average reward amount earned’^ ($) 

2,281 

1,838 

Percentage of earnings from student rewards 

58.1 

66.7 

Average reward amount earned, by domain ($) 

Education 

1,287 

1,278 

Health 

617 

284 

Workforce 

1,140 

1,337 

Family picked up coupon book (%) 

95.5 

89.5 

Family earned a reward in® (%) 

Year 1 but not Year 2 

2.3 

5.8 

Year 2 but not Year 1 

5.4 

4.0 

Year 1 and Year 2 

89.5 

84.4 


Sample size 6H 190 

(continued) 
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Appendix Table D,2 (continued) 


SOURCES: MDRC calculations using CAS's Family Rewards 2.0 program data 
and Seedco's Family Rewards 1.0 program data. 

NOTES: Sample sizes may vary because of missing values. NA = not applicable. 

Year 2 in Family Rewards 2.0 covers reward activities that occurred between 
September 2012 and August 2013. Year 2 in Family Rewards 1.0 covers reward 
activities that occurred between September 2008 and August 2009. 

Measures for Family Rewards 1.0 are calculated using only rewards 
comparable to those in the Family Rewards 2.0 program. These include rewards 
for Regents exams, attendance, and PSAT exams for high school students; annual 
physical and biannual dental visits for all family members; and full-time 
employment and employment training rewards for adults. 

Italics indicate outcomes calculated for a subset of the full sample. 

^Includes families from the Family Rewards 1.0 sample who were receiving 
TANF or food stamp benefits and living in the Bronx at the time of enrollment, 
and who had a high school student in the household in Year 2. 

'’College preparatory exams eligible for rewards included the PSAT for Family 
Rewards 1.0 and the SAT and ACT for Family Rewards 2.0. 

‘’The lowest amount earned in Year 2 in the 2.0 sample was $38 and the highest 
amount was $7,250; in the 1.0 TANF/food stamp sample the lowest amount 
earned was $100 and the highest amount earned was $6,050. 

‘'Percentage of earnings from student rewards is calculated using education 
rewards earned by high school students among families who earned rewards and 
had students who earned education rewards in Year 2. 

‘’Includes families from the Family Rewards 1.0 sample who were receiving 
TANF or food stamp benefits and living in the Bronx at the time of enrollment, 
and who had a high school student in Year 1 and Year 2. 
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About MDRC 


MDRC is a nonprofit, nonpartisan social and education policy research organization dedicated 
to learning what works to improve the well-being of low-income people. Through its research 
and the active communication of its findings, MDRC seeks to enhance the effectiveness of so- 
cial and education policies and programs. 

Founded in 1974 and located in New York City and Oakland, California, MDRC is best known 
for mounting rigorous, large-scale, real-world tests of new and existing policies and programs. 
Its projects are a mix of demonstrations (field tests of promising new program approaches) and 
evaluations of ongoing government and community initiatives. MDRC’s staff bring an unusual 
combination of research and organizational experience to their work, providing expertise on the 
latest in qualitative and quantitative methods and on program design, development, implementa- 
tion, and management. MDRC seeks to learn not just whether a program is effective but also 
how and why the program’s effects occur. In addition, it tries to place each project’s findings in 
the broader context of related research — in order to build knowledge about what works across 
the social and education policy fields. MDRC’s findings, lessons, and best practices are proac- 
tively shared with a broad audience in the policy and practitioner community as well as with the 
general public and the media. 

Over the years, MDRC has brought its unique approach to an ever-growing range of policy are- 
as and target populations. Once known primarily for evaluations of state welfare-to-work pro- 
grams, today MDRC is also studying public school reforms, employment programs for ex- 
offenders and people with disabilities, and programs to help low-income students succeed in 
college. MDRC’s projects are organized into five areas: 

• Promoting Family Well-Being and Children’s Development 

• Improving Public Education 

• Raising Academic Achievement and Persistence in College 

• Supporting Low-Wage Workers and Communities 

• Overcoming Barriers to Employment 

Working in almost every state, all of the nation’s largest cities, and Canada and the United 
Kingdom, MDRC conducts its projects in partnership with national, state, and local govern- 
ments, public school systems, community organizations, and numerous private philanthropies. 



